Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
SmartMonit: Real-time Big Data Monitoring System
Newcastle University. Bartin University.
Newcastle University. Taibah University.
Newcastle University.
Newcastle University.
Show others and affiliations
2019 (English)In: 2019 IEEE 38th International Symposium on Reliable Distributed Systems: SRDS 2019, IEEE, 2019, p. 357-359Conference paper, Published paper (Other academic)
Abstract [en]

Modern big data processing systems are becoming very complex in terms of large-scale, high-concurrency and multiple talents. Thus, many failures and performance reductions only happen at run-time and are very difficult to capture. Moreover, some issues may only be triggered when some components are executed. To analyze the root cause of these types of issues, we have to capture the dependencies of each component in real-time. In this paper, we propose SmartMonit, a real-time big data monitoring system, which collects infrastructure information such as the process status of each task. At the same time, we develop a real-time stream processing framework to analyze the coordination among the tasks and the infrastructures. This coordination information is essential for troubleshooting the reasons for failures and performance reduction, especially the ones propagated from other causes.

Place, publisher, year, edition, pages
IEEE, 2019. p. 357-359
Series
Symposium on Reliable Distributed Systems, ISSN 1060-9857, E-ISSN 2575-8462
Keywords [en]
Hadoop, MapReduce, Monitoring
National Category
Computer and Information Sciences
Research subject
Pervasive Mobile Computing
Identifiers
URN: urn:nbn:se:ltu:diva-76617DOI: 10.1109/SRDS47363.2019.00049ISI: 000720089500038Scopus ID: 2-s2.0-85084118011OAI: oai:DiVA.org:ltu-76617DiVA, id: diva2:1368025
Conference
The 38th International Symposium on Reliable Distributed Systems (SRDS 2019) Lyon, France, OCT 1-4, 2019
Note

ISBN för värdpublikation: 978-1-7281-4222-7, 978-1-7281-4223-4

Available from: 2019-11-05 Created: 2019-11-05 Last updated: 2025-02-18Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Mitra, Karan

Search in DiVA

By author/editor
Mitra, Karan
By organisation
Computer Science
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 260 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf