Turbine: A Distributed-memory Dataflow Engine for High Performance Many-task Applications.
Saved in:
| Title: | Turbine: A Distributed-memory Dataflow Engine for High Performance Many-task Applications. |
|---|---|
| Authors: | Wozniak, Justin M.1 wozniak@mcs.anl.gov, Armstrong, Timothy G.2 tga@uchicago.edu, Maheshwari, Ketan1 ketan@mcs.anl.gov, Lusk, Ewing L.1 lusk@mcs.anl.gov, Katz, Daniel S.3 d.katz@ieee.org, Wilde, Michael1 wilde@mcs.anl.gov, Foster, Ian T.1 foster@mcs.anl.gov |
| Source: | Fundamenta Informaticae. 2013, Vol. 128 Issue 3, p337-366. 30p. |
| Subjects: | Turbines, Distributed shared memory, High performance computing, Application software, Computer programming, Computer performance, Data flow computing |
| Abstract: | Efficiently utilizing the rapidly increasing concurrency of multi-petaflop computing systems is a significant programming challenge. One approach is to structure applications with an upper layer of many loosely coupled coarse-grained tasks, each comprising a tightly-coupled parallel function or program. 'Many-task' programming models such as functional parallel dataflow may be used at the upper layer to generate massive numbers of tasks, each of which generates significant tightly coupled parallelism at the lower level through multithreading, message passing, and/or partitioned global address spaces. At large scales, however, the management of task distribution, data dependencies, and intertask data movement is a significant performance challenge. In this work, we describe Turbine, a new highly scalable and distributed many-task dataflow engine. Turbine executes a generalized many-task intermediate representation with automated self-distribution and is scalable to multi-petaflop infrastructures. We present here the architecture of Turbine and its performance on highly concurrent systems. [ABSTRACT FROM AUTHOR] |
| Copyright of Fundamenta Informaticae is the property of Polskie Towarzystwo Matematyczne and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 92009891 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Turbine: A Distributed-memory Dataflow Engine for High Performance Many-task Applications. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Wozniak%2C+Justin+M%2E%22">Wozniak, Justin M.</searchLink><relatesTo>1</relatesTo><i> wozniak@mcs.anl.gov</i><br /><searchLink fieldCode="AR" term="%22Armstrong%2C+Timothy+G%2E%22">Armstrong, Timothy G.</searchLink><relatesTo>2</relatesTo><i> tga@uchicago.edu</i><br /><searchLink fieldCode="AR" term="%22Maheshwari%2C+Ketan%22">Maheshwari, Ketan</searchLink><relatesTo>1</relatesTo><i> ketan@mcs.anl.gov</i><br /><searchLink fieldCode="AR" term="%22Lusk%2C+Ewing+L%2E%22">Lusk, Ewing L.</searchLink><relatesTo>1</relatesTo><i> lusk@mcs.anl.gov</i><br /><searchLink fieldCode="AR" term="%22Katz%2C+Daniel+S%2E%22">Katz, Daniel S.</searchLink><relatesTo>3</relatesTo><i> d.katz@ieee.org</i><br /><searchLink fieldCode="AR" term="%22Wilde%2C+Michael%22">Wilde, Michael</searchLink><relatesTo>1</relatesTo><i> wilde@mcs.anl.gov</i><br /><searchLink fieldCode="AR" term="%22Foster%2C+Ian+T%2E%22">Foster, Ian T.</searchLink><relatesTo>1</relatesTo><i> foster@mcs.anl.gov</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Fundamenta+Informaticae%22">Fundamenta Informaticae</searchLink>. 2013, Vol. 128 Issue 3, p337-366. 30p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Turbines%22">Turbines</searchLink><br /><searchLink fieldCode="DE" term="%22Distributed+shared+memory%22">Distributed shared memory</searchLink><br /><searchLink fieldCode="DE" term="%22High+performance+computing%22">High performance computing</searchLink><br /><searchLink fieldCode="DE" term="%22Application+software%22">Application software</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+programming%22">Computer programming</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+performance%22">Computer performance</searchLink><br /><searchLink fieldCode="DE" term="%22Data+flow+computing%22">Data flow computing</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Efficiently utilizing the rapidly increasing concurrency of multi-petaflop computing systems is a significant programming challenge. One approach is to structure applications with an upper layer of many loosely coupled coarse-grained tasks, each comprising a tightly-coupled parallel function or program. 'Many-task' programming models such as functional parallel dataflow may be used at the upper layer to generate massive numbers of tasks, each of which generates significant tightly coupled parallelism at the lower level through multithreading, message passing, and/or partitioned global address spaces. At large scales, however, the management of task distribution, data dependencies, and intertask data movement is a significant performance challenge. In this work, we describe Turbine, a new highly scalable and distributed many-task dataflow engine. Turbine executes a generalized many-task intermediate representation with automated self-distribution and is scalable to multi-petaflop infrastructures. We present here the architecture of Turbine and its performance on highly concurrent systems. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Fundamenta Informaticae is the property of Polskie Towarzystwo Matematyczne and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=92009891 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.3233/FI-2013-949 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 30 StartPage: 337 Subjects: – SubjectFull: Turbines Type: general – SubjectFull: Distributed shared memory Type: general – SubjectFull: High performance computing Type: general – SubjectFull: Application software Type: general – SubjectFull: Computer programming Type: general – SubjectFull: Computer performance Type: general – SubjectFull: Data flow computing Type: general Titles: – TitleFull: Turbine: A Distributed-memory Dataflow Engine for High Performance Many-task Applications. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Wozniak, Justin M. – PersonEntity: Name: NameFull: Armstrong, Timothy G. – PersonEntity: Name: NameFull: Maheshwari, Ketan – PersonEntity: Name: NameFull: Lusk, Ewing L. – PersonEntity: Name: NameFull: Katz, Daniel S. – PersonEntity: Name: NameFull: Wilde, Michael – PersonEntity: Name: NameFull: Foster, Ian T. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 08 Text: 2013 Type: published Y: 2013 Identifiers: – Type: issn-print Value: 01692968 Numbering: – Type: volume Value: 128 – Type: issue Value: 3 Titles: – TitleFull: Fundamenta Informaticae Type: main |
| ResultId | 1 |
