sharding - MongoDB shard key as (ObjectId, ObjectId, RandomKey). Unbalanced collections -

-

i trying shard collection approximately 6m documents. following details sharded cluster

  1. mongod version 2.6.7, 2 shards, 40 % writes, 60% reads.

  2. my database has collection events around 6m documents. normal document looks below:

    {

          _id         : objectid,       sector_id   : objectid,       subsector_id: objectid,       .       .       .        many event specific fields go here       .       .        created_at: date,       updated_at: date,       uid       : 16digitrandomkey

    }

  3. each sector has multiple (1,2, ..100) subsectors , each subsector has multiple events. there 10000 such sectors, 30000 subsectors , 6m events. numbers keep growing.

  4. the normal read query includes sector_id, subsector_id. every write operation includes sector_id, subsector_id, uid (randomly generated unique key) , rest of data.

  5. i tried/considered following shard keys , results described below:

    a. _id:hashed --> not provide query isolation, reason: _id not passed read query.

    b. sector_id :1, subsector_id:1, uid:1 --> strange distribution: few sectors old objectid goes shard 1, few sectors having sector_id of mid age(objectid) balanced , equally distributed among both shards. few sectors recent objectid stays on shard 0.

    c. subsector_id: hashed --> results same shard key b.

    d. subsector_id:1, uid:1 --> same b.

    e. subsector_id:hashed, uid:1 --> can not create such index

    f. uid:1 --> writes distributed no query isolation

    what may reason uneven distribution? can right shard key based upon given data.

i see expected behaviour astro, sectorids , subsectorids objectid type contains timestamp first 4 bytes monotonic in nature , go same chunk (and hence same shard) failed provide randomness pointed in point (b).

the best way choose shard key key has business meaning (unlike objectid field) , should mixed hash suffix ensure random mix on equal distribution. if have sectorname , subsectorname pls try out , let know if working using that.

you may consider link choose right shard key.

mongodb shard date on single machine

-$


Comments

Post a Comment

Popular posts from this blog

Android : Making Listview full screen -

-
<linearlayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical" tools:context=".mainactivity"> <edittext android:layout_margintop="5dp" android:layout_width="match_parent" android:layout_height="50dp" android:background="@drawable/location_edittext" android:id="@+id/locationeditext" android:textsize="15sp" android:textcolor="#999" android:paddingleft="15dp" /> <com.example.ravi_gupta.slider.viewpagercustomduration android:layout_width="wrap_content" android:layout_height="0dp" android:id="@+id/viewpager" android:scrollbarstyle="outsideoverlay" android:layout_weight="0.5&quo
Read more

javascript - Parse JSON from the body of the POST -

-
i'm using cameratag.com record video on website. service can send json response server can save data database. don't know how proceed here: the documentation states please note json repsonse passed in body of post , not form parameters. here example of how parse response in ruby: data = json.parse(request.raw_post) video_uuid = data["uuid"] but don't know how data on javascript. i'm not experienced in js please bare me :) post sample: { "uuid": "00012710-4b65-0131-ffe6-22000a499ea4", "camera_uuid": "98373a20-79ee-0130-3e42-1231390fcc11", "created_at": "2013-12-20t05:25:02.000z", "percent_complete": 100, "short_code": "kktb4q", "metadata": { }, "state": "published", "recorded_from": "http://singwho.com", "publish_type": null, "formats": [ { "name&qu
Read more

javascript - Chrome Extension: Interacting with iframe embedded within popup -

-
i writing chrome extension needs interact urls user not have open. therefore using hidden iframes embedded within popup, , attempting click button within iframe. however, receiving same origin policy error. know possible extension interact iframes of different domain via content scripts when iframe on tab user has open, not sure if possible use content scripts interact iframes directly in popup. here code: manifest.json "content_scripts": [ { "js": [ "bin/jquery.min.js", "interaction.js" ], "all_frames": true, "run_at": "document_start", "matches": [ "http://*/*", "https://*/*" ] }], "permissions": [ "activetab", "tabs", "http://*/", "https://*/" ], interaction.js $(document).ready(function() { $('div#iframes').append("<iframe id='shop' src=
Read more