Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
SmartMonit: Real-time Big Data Monitoring System
Newcastle University. Bartin University.
Newcastle University. Taibah University.
Newcastle University.
Newcastle University.
Visa övriga samt affilieringar
2019 (Engelska)Ingår i: 2019 IEEE 38th International Symposium on Reliable Distributed Systems: SRDS 2019, IEEE, 2019, s. 357-359Konferensbidrag, Publicerat paper (Övrigt vetenskapligt)
Abstract [en]

Modern big data processing systems are becoming very complex in terms of large-scale, high-concurrency and multiple talents. Thus, many failures and performance reductions only happen at run-time and are very difficult to capture. Moreover, some issues may only be triggered when some components are executed. To analyze the root cause of these types of issues, we have to capture the dependencies of each component in real-time. In this paper, we propose SmartMonit, a real-time big data monitoring system, which collects infrastructure information such as the process status of each task. At the same time, we develop a real-time stream processing framework to analyze the coordination among the tasks and the infrastructures. This coordination information is essential for troubleshooting the reasons for failures and performance reduction, especially the ones propagated from other causes.

Ort, förlag, år, upplaga, sidor
IEEE, 2019. s. 357-359
Serie
Symposium on Reliable Distributed Systems, ISSN 1060-9857, E-ISSN 2575-8462
Nyckelord [en]
Hadoop, MapReduce, Monitoring
Nationell ämneskategori
Data- och informationsvetenskap
Forskningsämne
Distribuerade datorsystem
Identifikatorer
URN: urn:nbn:se:ltu:diva-76617DOI: 10.1109/SRDS47363.2019.00049ISI: 000720089500038Scopus ID: 2-s2.0-85084118011OAI: oai:DiVA.org:ltu-76617DiVA, id: diva2:1368025
Konferens
The 38th International Symposium on Reliable Distributed Systems (SRDS 2019) Lyon, France, OCT 1-4, 2019
Anmärkning

ISBN för värdpublikation: 978-1-7281-4222-7, 978-1-7281-4223-4

Tillgänglig från: 2019-11-05 Skapad: 2019-11-05 Senast uppdaterad: 2025-02-18Bibliografiskt granskad

Open Access i DiVA

Fulltext saknas i DiVA

Övriga länkar

Förlagets fulltextScopus

Person

Mitra, Karan

Sök vidare i DiVA

Av författaren/redaktören
Mitra, Karan
Av organisationen
Datavetenskap
Data- och informationsvetenskap

Sök vidare utanför DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetricpoäng

doi
urn-nbn
Totalt: 262 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf