How to separate mongodb data files from the journal?

How should I set up mongodb cluster to handle 20K+ simultaneous connections?

• My application uses MongoDB as database. We are expecting 20K+ simultaneous connections to mongodb cluster. How should I config the server if I want to run the mongodb on 20 servers and shard the cluster 20 ways? Here is what I've done so far: On each of my 20 servers, I have one mongos (router) running on port 30000, and on 3 servers I run mongo config servers on port 20000. Then on each server, I run 3 instances of mongod. One of the is the primary. In order words, I have 20 mongos, 3 mongo-config, 60 mongod servers (20 primary and 40 replica). Then in my application (which also run on each server and connect to the localhost:30000 mongos), I set the mongoOptions such that the connectionsPerHost = 1000. 10-15 minutes after all services start, some of them became no longer ssh-able. These servers are still ping-able. I suspect there were too many connections, and it caused the server to die. My own analysis is as follows: 1K connections per connection pool means for each shard's primary, it will have 1K * 20 (shards) = 20K simultaneous connections open. A few servers will probably have more than one primary running on it, which will double or triple the number of connections to 60K. Somehow mongod cannot handle these many connections although I changed my system settings to allow each process to open way more files. Here are what 'ulimit -a' shows: --------------------------------------------------- core file size          (blocks, -c) 0 data seg size           (kbytes, -d) unlimited scheduling priority             (-e) 20 file size               (blocks, -f) unlimited pending signals                 (-i) 16382 max locked memory       (kbytes, -l) 64000000 max memory size         (kbytes, -m) unlimited open files                      (-n) 320000 pipe size            (512 bytes, -p) 8 POSIX message queues     (bytes, -q) 819200 real-time priority              (-r) 0 stack size              (kbytes, -s) 8192 cpu time               (seconds, -t) unlimited max user processes              (-u) unlimited virtual memory          (kbytes, -v) unlimited file locks                      (-x) unlimited BTW, I didn't specify --maxConns when I start up mongod/mongos, I also didn't change MONGO.POOLSIZE. COULD ANYONE HELP ME OUT? Thank you. A side question: if my reasoning is correct, the total number of simultaneous connection requirement will be posed on each primary, which doesn't seem right to me, it almost means mongodb cluster is not scalable at all. Someone tell me I'm wrong please?

• Answer:

  In a sharded configuration each mongos will have 1 incoming connection from the client but may need 1 outgoing connection to each shard. This means that the possible number of open connections that a mongos server requires could be (1 + (N*M) * C) where N = number of shards, M = number of replicaset nodes, and C = number of client connections. Its not clear who you are laying out the mongo process across the servers, typically you will need to physically separate them. See this article http://www.mongodb.org/display/DOCS/Simple+Initial+Sharding+Architecture The mongos processes can reside on your application server machines, so that may gain you back some resources,