In Hive:
When I was studying how to run Hive on Amazon Elastic MapReduce, I encountered one AWS online tutorial which introduced a custom Serde (Serializer-deserializer) to parse JSON data. According to the tutorial, the jar file for the serializer-deserializer could be included at a .q Hive scripts by placing:
add jar s3://elasticmapreduce/samples/hive-ads/libs/jsonserde.jar;
at the top of the scripts.
Code snippet below illustrates a simple example:
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgC9aqdIk8nt-p-5CcZB8WSASLPhudBAJedMRBkYWz_JnVsyrb2A7yTOcLsew_KqTNRnnbOf3MnVCemRJLUalk_rFeYXQHH9YefHbdPNpNLpSyM1Ynz5OLNwkXadX8tNJzkidD6N4I4MhI/s400/hive.png)
The four words assigned to ‘paths’ are the four corresponding field names used by the JSON source data for the four fields in the Hive table declared above. Notice that the last field ‘user.id’ demonstrates how to retrieve nested format data.
In Java:
Online tutorial posted on http://www.java2blog.com/ contains detailed examples for the two most widely used Java libraries for JSON processing: JSON.simple and GSON. I followed these instructions during development.
In Python:
I used json, the JSON encoder and decoder in Python.
The following code snippet does the exactly same job as compared to the Hive script example above.
The json.loads() method was used, details could be found at here.
No comments:
Post a Comment