hadoop - Does Spark not support arraylist when writing to elasticsearch? -

-

i have following structure:

mylist = [{"key1":"val1"}, {"key2":"val2"}] myrdd = value_counts.map(lambda item: ('key', {      'field': somelist  }))

i error: 15/02/10 15:54:08 info scheduler.tasksetmanager: lost task 1.0 in stage 2.0 (tid 6) on executor ip-10-80-15-145.ec2.internal: org.apache.spark.sparkexception (data of type java.util.arraylist cannot used) [duplicate 1] 

rdd.saveasnewapihadoopfile(              path='-',              outputformatclass="org.elasticsearch.hadoop.mr.esoutputformat",              keyclass="org.apache.hadoop.io.nullwritable",              valueclass="org.elasticsearch.hadoop.mr.linkedmapwritable",              conf={          "es.nodes" : "localhost",          "es.port" : "9200",          "es.resource" : "mboyd/mboydtype"      })

what want document end when written es is:

{ field:[{"key1":"val1"}, {"key2":"val2"}] }

just had problem, , solution passes converting lists tuples . converting json same.


Comments

Post a Comment

Popular posts from this blog

Android : Making Listview full screen -

-
<linearlayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical" tools:context=".mainactivity"> <edittext android:layout_margintop="5dp" android:layout_width="match_parent" android:layout_height="50dp" android:background="@drawable/location_edittext" android:id="@+id/locationeditext" android:textsize="15sp" android:textcolor="#999" android:paddingleft="15dp" /> <com.example.ravi_gupta.slider.viewpagercustomduration android:layout_width="wrap_content" android:layout_height="0dp" android:id="@+id/viewpager" android:scrollbarstyle="outsideoverlay" android:layout_weight="0.5&quo
Read more

javascript - Parse JSON from the body of the POST -

-
i'm using cameratag.com record video on website. service can send json response server can save data database. don't know how proceed here: the documentation states please note json repsonse passed in body of post , not form parameters. here example of how parse response in ruby: data = json.parse(request.raw_post) video_uuid = data["uuid"] but don't know how data on javascript. i'm not experienced in js please bare me :) post sample: { "uuid": "00012710-4b65-0131-ffe6-22000a499ea4", "camera_uuid": "98373a20-79ee-0130-3e42-1231390fcc11", "created_at": "2013-12-20t05:25:02.000z", "percent_complete": 100, "short_code": "kktb4q", "metadata": { }, "state": "published", "recorded_from": "http://singwho.com", "publish_type": null, "formats": [ { "name&qu
Read more

javascript - Chrome Extension: Interacting with iframe embedded within popup -

-
i writing chrome extension needs interact urls user not have open. therefore using hidden iframes embedded within popup, , attempting click button within iframe. however, receiving same origin policy error. know possible extension interact iframes of different domain via content scripts when iframe on tab user has open, not sure if possible use content scripts interact iframes directly in popup. here code: manifest.json "content_scripts": [ { "js": [ "bin/jquery.min.js", "interaction.js" ], "all_frames": true, "run_at": "document_start", "matches": [ "http://*/*", "https://*/*" ] }], "permissions": [ "activetab", "tabs", "http://*/", "https://*/" ], interaction.js $(document).ready(function() { $('div#iframes').append("<iframe id='shop' src=
Read more