How to use ConvertToWritableTypes in scala spark? -

-

i looking @ basicsavesequencefile , tried follow in scala

so had:

val input = seq(("coffee", 1), ("coffee", 2), ("pandas", 3)) val inputrdd = sc.parallelize(input) // no parallelizepairs

but when try:

val outputrdd = inputrdd.map(new converttowritabletypes()) // have no maptopair how write instead?

how use converttowritabletypes in scala spark?

right get:

error:(29, 38) type mismatch;  found   : sparkexamplewriteseqlzo.converttowritabletypes  required: ((string, int)) => ?         val outputrdd = inputrdd.map(new converttowritabletypes())                                      ^

so you're looking @ java version, should looking @ scala version api's different. example give, don't need maptopair, can use normal map without static class:

  import org.apache.hadoop.io.intwritable   import org.apache.hadoop.io.text    val input = seq(("coffee", 1), ("coffee", 2), ("pandas", 3))   val inputrdd = sc.parallelize(input)    val outputrdd = inputrdd.map(record => (new text(record._1), new intwritable(record._2)))

you don't need though, scala example linked shows you:

  val data = sc.parallelize(list(("holden", 3), ("kay", 6), ("snail", 2)))   data.saveassequencefile(outputfile)

Comments

Post a Comment

Popular posts from this blog

Android : Making Listview full screen -

-
<linearlayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical" tools:context=".mainactivity"> <edittext android:layout_margintop="5dp" android:layout_width="match_parent" android:layout_height="50dp" android:background="@drawable/location_edittext" android:id="@+id/locationeditext" android:textsize="15sp" android:textcolor="#999" android:paddingleft="15dp" /> <com.example.ravi_gupta.slider.viewpagercustomduration android:layout_width="wrap_content" android:layout_height="0dp" android:id="@+id/viewpager" android:scrollbarstyle="outsideoverlay" android:layout_weight="0.5&quo
Read more

javascript - Parse JSON from the body of the POST -

-
i'm using cameratag.com record video on website. service can send json response server can save data database. don't know how proceed here: the documentation states please note json repsonse passed in body of post , not form parameters. here example of how parse response in ruby: data = json.parse(request.raw_post) video_uuid = data["uuid"] but don't know how data on javascript. i'm not experienced in js please bare me :) post sample: { "uuid": "00012710-4b65-0131-ffe6-22000a499ea4", "camera_uuid": "98373a20-79ee-0130-3e42-1231390fcc11", "created_at": "2013-12-20t05:25:02.000z", "percent_complete": 100, "short_code": "kktb4q", "metadata": { }, "state": "published", "recorded_from": "http://singwho.com", "publish_type": null, "formats": [ { "name&qu
Read more

javascript - Chrome Extension: Interacting with iframe embedded within popup -

-
i writing chrome extension needs interact urls user not have open. therefore using hidden iframes embedded within popup, , attempting click button within iframe. however, receiving same origin policy error. know possible extension interact iframes of different domain via content scripts when iframe on tab user has open, not sure if possible use content scripts interact iframes directly in popup. here code: manifest.json "content_scripts": [ { "js": [ "bin/jquery.min.js", "interaction.js" ], "all_frames": true, "run_at": "document_start", "matches": [ "http://*/*", "https://*/*" ] }], "permissions": [ "activetab", "tabs", "http://*/", "https://*/" ], interaction.js $(document).ready(function() { $('div#iframes').append("<iframe id='shop' src=
Read more