Bayt.com

Start networking and exchanging professional insights

Register now or log in to join your professional community.

Follow

Can we Use Hadoop Technology to process data in a live streaming environment? For example to process data feed coming from Flight auto log system?

Machine Learning SQL SAS
user-image
Question added by Abrar Mohd , Consulting Engineer , CISCO SYSTEMS
Date Posted: 2014/02/27
Upvote (0) Views (9) Followers (2) Answers (2) Report Question
Write an Answer
Register now or log in to answer.
Sudheer J
by Sudheer J , Lead , Ford

Yes Apache Flume helps in your case .

 

Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of streaming data into the Hadoop Distributed File System (HDFS).

The use of Apache Flume is not only restricted to log data aggregation. Flume can be used to transport massive quantities of event data including but not limited to network traffic data, social-media-generated data, email messages and pretty much any data source possible.

Flume allows users to:

Stream data from multiple sources into Hadoop for analysis

Collect high-volume Web logs in real time

Guarantee data delivery

Scale horizontally to handle additional data volume

 

 

Once data is available in HDFS , you can write MapReduce , Hive SQL queries on the top of  data to do analysis . Hope this helps !!

Gayasuddin Mohammed
by Gayasuddin Mohammed , Advocate , Practicing Law before High Court at Hyderabad

No. As per my knowledge Hadoop Technology is not for to process data in a live streaming environment. Thanks.

Write Your Answer
More Questions Like This

Do you need help in adding the right keywords to your CV? Let our CV writing experts help you.

Get Help