Here is a pretty wild suggestion
Convert all ngrams to 32-character MD5 keys
This table will hold all ngrams of any size (up to 255 characters), 1-gram, 2-gram, etc.
use test
DROP TABLE ngram_node;
DROP TABLE ngram_blackhole;
CREATE TABLE ngram_node
(
NGRAM_KEY CHAR(32) NOT NULL,
NGRAM_YEAR SMALLINT NOT NULL,
M_COUNT SMALLINT NOT NULL,
P_COUNT SMALLINT NOT NULL,
V_COUNT SMALLINT NOT NULL,
PRIMARY KEY (NGRAM_KEY,NGRAM_YEAR)
) ENGINE=MyISAM
PARTITION BY KEY(NGRAM_KEY)
PARTITIONS 256;
CREATE TABLE ngram_blackhole
(
NGRAM VARCHAR(255) NOT NULL,
NGRAM_YEAR SMALLINT NOT NULL,
M_COUNT SMALLINT NOT NULL,
P_COUNT SMALLINT NOT NULL,
V_COUNT SMALLINT NOT NULL
) ENGINE=BLACKHOLE;
DELIMITER $$
CREATE TRIGGER populate_ngram AFTER INSERT ON ngram_blackhole FOR EACH ROW
BEGIN
INSERT INTO ngram_node VALUES (MD5(NEW.NGRAM),NEW.NGRAM_YEAR,NEW.M_COUNT,NEW.P_COUNT,NEW.V_COUNT);
END; $$
DELIMITER ;
INSERT INTO ngram_blackhole VALUES
('rolando',1965,31,29,85),
('pamela',1971,33,21,86),
('dominique',1996,30,18,87),
('diamond',1998,13,28,88),
('rolando edwards',1965,31,29,85),
('pamela edwards',1971,33,21,86),
('dominique edwards',1996,30,18,87),
('diamond edwards',1998,13,28,88),
('rolando angel edwards',1965,31,29,85),
('pamela claricia edwards',1971,33,21,86),
('dominique sharlisee edwards',1996,30,18,87),
('diamond ashley edwards',1998,13,28,88);
SELECT * FROM ngram_node;
The reason I chose 256 partitions stems from the fact that MD5 return 16 different characters (all hexadecimal digits). First two bytes is 16 X 16, 256.
Here were the result in MySQL 5.5.11 on my Windows 7 Desktop
mysql> use test
Database changed
mysql> DROP TABLE ngram_node;
Query OK, 0 rows affected (0.22 sec)
mysql> DROP TABLE ngram_blackhole;
Query OK, 0 rows affected (0.11 sec)
mysql> CREATE TABLE ngram_node
-> (
-> NGRAM_KEY CHAR(32) NOT NULL,
-> NGRAM_YEAR SMALLINT NOT NULL,
-> M_COUNT SMALLINT NOT NULL,
-> P_COUNT SMALLINT NOT NULL,
-> V_COUNT SMALLINT NOT NULL,
-> PRIMARY KEY (NGRAM_KEY,NGRAM_YEAR)
-> ) ENGINE=MyISAM
-> PARTITION BY KEY(NGRAM_KEY)
-> PARTITIONS 256;
Query OK, 0 rows affected (0.36 sec)
mysql> CREATE TABLE ngram_blackhole
-> (
-> NGRAM VARCHAR(255) NOT NULL,
-> NGRAM_YEAR SMALLINT NOT NULL,
-> M_COUNT SMALLINT NOT NULL,
-> P_COUNT SMALLINT NOT NULL,
-> V_COUNT SMALLINT NOT NULL
-> ) ENGINE=BLACKHOLE;
Query OK, 0 rows affected (0.11 sec)
mysql> DELIMITER $$
mysql> CREATE TRIGGER populate_ngram AFTER INSERT ON ngram_blackhole FOR EACH ROW
-> BEGIN
-> INSERT INTO ngram_node VALUES (MD5(NEW.NGRAM),NEW.NGRAM_YEAR,NEW.M_COUNT,NEW.P_COUNT,NEW.V_COUNT);
-> END; $$
Query OK, 0 rows affected (0.05 sec)
mysql> DELIMITER ;
mysql> INSERT INTO ngram_blackhole VALUES
-> ('rolando',1965,31,29,85),
-> ('pamela',1971,33,21,86),
-> ('dominique',1996,30,18,87),
-> ('diamond',1998,13,28,88),
-> ('rolando edwards',1965,31,29,85),
-> ('pamela edwards',1971,33,21,86),
-> ('dominique edwards',1996,30,18,87),
-> ('diamond edwards',1998,13,28,88),
-> ('rolando angel edwards',1965,31,29,85),
-> ('pamela claricia edwards',1971,33,21,86),
-> ('dominique sharlisee edwards',1996,30,18,87),
-> ('diamond ashley edwards',1998,13,28,88);
Query OK, 12 rows affected (0.18 sec)
Records: 12 Duplicates: 0 Warnings: 0
mysql> SELECT * FROM ngram_node;
+----------------------------------+------------+---------+---------+---------+
| NGRAM_KEY | NGRAM_YEAR | M_COUNT | P_COUNT | V_COUNT |
+----------------------------------+------------+---------+---------+---------+
| 2ca237192aaac3b3a20ce0649351b395 | 1996 | 30 | 18 | 87 |
| 6f7fd3368170c562604f62fb4e92056d | 1965 | 31 | 29 | 85 |
| fb201333fef377917be714dabd3776d9 | 1971 | 33 | 21 | 86 |
| 4f79e21800ed6e30be4d1cb597f910c6 | 1971 | 33 | 21 | 86 |
| 9068e0de9f3fd674d4fa7cbc626e5888 | 1998 | 13 | 28 | 88 |
| 8a18abe90f2612827dc3a215fd1905d3 | 1965 | 31 | 29 | 85 |
| be60b431a46fcc7bf5ee4f7712993e3b | 1996 | 30 | 18 | 87 |
| c8adc38aa00759488b1d759aa8f91725 | 1996 | 30 | 18 | 87 |
| e80d4ab77eb18a4ca350157fd487d7e2 | 1965 | 31 | 29 | 85 |
| 669ffc150d1f875819183addfc842cab | 1971 | 33 | 21 | 86 |
| b685323e9de65080f733b53b2305da6e | 1998 | 13 | 28 | 88 |
| 75c6f03161d020201000414cd1501f9f | 1998 | 13 | 28 | 88 |
+----------------------------------+------------+---------+---------+---------+
12 rows in set (0.00 sec)
mysql>
Please notice I loaded 1-grams, 2-grams, and 3-grams into the same table but you have no clue which MD5 belongs to the ngram. Thus, all ngrams can retrofit into this table. Just remember to insert into ngram_blackhole and the rest is done for you.
You must query the ngram_node table using the MD5() of the ngram no matter which ngram.
mysql> select * from ngram_node where ngram_key=MD5('rolando edwards');
+----------------------------------+------------+---------+---------+---------+
| NGRAM_KEY | NGRAM_YEAR | M_COUNT | P_COUNT | V_COUNT |
+----------------------------------+------------+---------+---------+---------+
| 6f7fd3368170c562604f62fb4e92056d | 1965 | 31 | 29 | 85 |
+----------------------------------+------------+---------+---------+---------+
1 row in set (0.05 sec)
If you wish to separate 1-grams, 2-grams, and 3-grams into separate repositories, just create another table, another blackhole, and another stored procedure on the blackhole.
Give it a Try !!!
UPDATE
I saw two typos in my answer. I had ENGINE=BLACKOLE. I changed it to ENGINE=BLACKHOLE. I reran everything. It turned out the same result !!!
Also, if your ngrams are longer than 255 (if you are doing 7-grams or 8-grams) just increase the VARCHAR size of the NGRAM column in the ngram_blackhole table.