sharding - MongoDB shard key as (ObjectId, ObjectId, RandomKey). Unbalanced collections -

-

i trying shard collection approximately 6m documents. following details sharded cluster

  1. mongod version 2.6.7, 2 shards, 40 % writes, 60% reads.

  2. my database has collection events around 6m documents. normal document looks below:

    {

          _id         : objectid,       sector_id   : objectid,       subsector_id: objectid,       .       .       .        many event specific fields go here       .       .        created_at: date,       updated_at: date,       uid       : 16digitrandomkey

    }

  3. each sector has multiple (1,2, ..100) subsectors , each subsector has multiple events. there 10000 such sectors, 30000 subsectors , 6m events. numbers keep growing.

  4. the normal read query includes sector_id, subsector_id. every write operation includes sector_id, subsector_id, uid (randomly generated unique key) , rest of data.

  5. i tried/considered following shard keys , results described below:

    a. _id:hashed --> not provide query isolation, reason: _id not passed read query.

    b. sector_id :1, subsector_id:1, uid:1 --> strange distribution: few sectors old objectid goes shard 1, few sectors having sector_id of mid age(objectid) balanced , equally distributed among both shards. few sectors recent objectid stays on shard 0.

    c. subsector_id: hashed --> results same shard key b.

    d. subsector_id:1, uid:1 --> same b.

    e. subsector_id:hashed, uid:1 --> can not create such index

    f. uid:1 --> writes distributed no query isolation

    what may reason uneven distribution? can right shard key based upon given data.

i see expected behaviour astro, sectorids , subsectorids objectid type contains timestamp first 4 bytes monotonic in nature , go same chunk (and hence same shard) failed provide randomness pointed in point (b).

the best way choose shard key key has business meaning (unlike objectid field) , should mixed hash suffix ensure random mix on equal distribution. if have sectorname , subsectorname pls try out , let know if working using that.

you may consider link choose right shard key.

mongodb shard date on single machine

-$


Comments

Post a Comment

Popular posts from this blog

Android : Making Listview full screen -

-
<linearlayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical" tools:context=".mainactivity"> <edittext android:layout_margintop="5dp" android:layout_width="match_parent" android:layout_height="50dp" android:background="@drawable/location_edittext" android:id="@+id/locationeditext" android:textsize="15sp" android:textcolor="#999" android:paddingleft="15dp" /> <com.example.ravi_gupta.slider.viewpagercustomduration android:layout_width="wrap_content" android:layout_height="0dp" android:id="@+id/viewpager" android:scrollbarstyle="outsideoverlay" android:layout_weight="0.5...
Read more

javascript - Parse JSON from the body of the POST -

-
i'm using cameratag.com record video on website. service can send json response server can save data database. don't know how proceed here: the documentation states please note json repsonse passed in body of post , not form parameters. here example of how parse response in ruby: data = json.parse(request.raw_post) video_uuid = data["uuid"] but don't know how data on javascript. i'm not experienced in js please bare me :) post sample: { "uuid": "00012710-4b65-0131-ffe6-22000a499ea4", "camera_uuid": "98373a20-79ee-0130-3e42-1231390fcc11", "created_at": "2013-12-20t05:25:02.000z", "percent_complete": 100, "short_code": "kktb4q", "metadata": { }, "state": "published", "recorded_from": "http://singwho.com", "publish_type": null, "formats": [ { "name...
Read more

javascript - How to Hide Date Menu from Datepicker in yii2 -

-
basically in datetimepicker widget date choose first time, how can hide date option widget , filter according time. tried not working want hide remove button [ 'attribute'=>'time', 'label'=>'time', 'format'=>'raw', 'value'=>function($data){ return date('h:i:s',strtotime($data->time)); }, 'filter'=> datetimepicker::widget([ 'model' => $searchmodel, 'attribute' => 'time', 'clientoptions' => [ 'autoclose' => true, 'format' => 'hh:ii', 'showon' =...
Read more