I have a field in my collection which is a String in nature. When i run aggregation for this field in data it takes considerable time in aggregation. Total 6 million records are in collection which has almost 500k different value for this field.
I am in impression if a field is numeric with similar scenario then aggregation on that field will be very fast.
So i planned to save the string as unique integer value by converting it java biginteger but mongo does not support that. Could you please suggest me what could be the best possible way for this.
This is the data we want to aggregate
{
"_id" : ObjectId("54d7ac5c3848e23f6f471b9f"),
"articleId" : [
{
"articleId" : "17084694",
"articleCount" : 1
}
],
"host" : "abc.com",
"count" : NumberLong(1),
"cat" : "test test",
"catIds" : [
"-2128958273",
"2886704",
"5912641"
],
"timeStamp" : NumberLong("1423420500000"),
"interval" : 1,
"tags" : "magnetic fields"
}
{
"_id" : ObjectId("54d7ac5c3848e23f6f471b9f"),
"articleId" : [
{
"articleId" : "17084694",
"articleCount" : 1
}
{
"articleId" : "17084695",
"articleCount" : 1
}
],
"host" : "abc.com",
"count" : NumberLong(1),
"cat" : "test test",
"catIds" : [
"-2128958273",
"2886704",
"5912641"
],
"timeStamp" : NumberLong("1423420500000"),
"interval" : 1,
"tags" : "magnetic fields"
}
{
"_id" : ObjectId("54d7ac5c3848e23f6f471b9f"),
"articleId" : [
{
"articleId" : "17084695",
"articleCount" : 1
}
{
"articleId" : "17084696",
"articleCount" : 1
}
],
"host" : "abc.com",
"count" : NumberLong(1),
"cat" : "test test",
"catIds" : [
"-2128958273",
"2886704",
"5912641"
],
"timeStamp" : NumberLong("1423420500000"),
"interval" : 1,
"tags" : "taps records"
}
Result will be
{
"_id" : ObjectId("54d7ac5c3848e23f6f471b9f"),
"articleId" : [
{
"articleId" : "17084694",
"articleCount" : 2
}
{
"articleId" : "17084695",
"articleCount" : 1
}
],
"host" : "abc.com",
"count" : NumberLong(2),
"cat" : "test test",
"catIds" : [
"-2128958273",
"2886704",
"5912641"
],
"timeStamp" : NumberLong("1423420500000"),
"interval" : 1,
"tags" : "magnetic fields"
}
{
"_id" : ObjectId("54d7ac5c3848e23f6f471b9f"),
"articleId" : [
{
"articleId" : "17084695",
"articleCount" : 1
}
{
"articleId" : "17084696",
"articleCount" : 1
}
],
"host" : "abc.com",
"count" : NumberLong(1),
"cat" : "test test",
"catIds" : [
"-2128958273",
"2886704",
"5912641"
],
"timeStamp" : NumberLong("1423420500000"),
"interval" : 1,
"tags" : "taps records"
} As you see the aggregation is on tags field where two things are added one is count second artcileCount for same tags. IT takes a lot of time. Hash i was referring to keep tag string as some integer form and aggregate that