0

I have database with two tables: logs & src_info

mysql> describe logs;
+-----------+-------------+------+-----+---------+----------------+
| Field     | Type        | Null | Key | Default | Extra          |
+-----------+-------------+------+-----+---------+----------------+
| id        | int(11)     | NO   | PRI | NULL    | auto_increment |
| sessionid | int(8)      | NO   |     | NULL    |                |
| date      | date        | NO   |     | NULL    |                |
| time      | time(6)     | NO   |     | NULL    |                |
| src       | varchar(15) | NO   |     | NULL    |                |
| dst       | varchar(15) | NO   |     | NULL    |                |
| dstport   | int(6)      | NO   |     | NULL    |                |
| proto     | varchar(6)  | NO   |     | NULL    |                |
| rcvdbyte  | int(24)     | NO   |     | NULL    |                |
| sentbyte  | int(24)     | NO   |     | NULL    |                |
+-----------+-------------+------+-----+---------+----------------+
10 rows in set (0.00 sec)

AND:

mysql> describe src_ipinfo;
+----------+--------------+------+-----+---------+----------------+
| Field    | Type         | Null | Key | Default | Extra          |
+----------+--------------+------+-----+---------+----------------+
| id       | int(11)      | NO   | PRI | NULL    | auto_increment |
| src      | varchar(15)  | NO   |     | NULL    |                |
| hostname | varchar(50)  | NO   |     | NULL    |                |
| city     | varchar(50)  | NO   |     | NULL    |                |
| region   | varchar(50)  | NO   |     | NULL    |                |
| country  | varchar(2)   | NO   |     | NULL    |                |
| org      | varchar(150) | NO   |     | NULL    |                |
+----------+--------------+------+-----+---------+----------------+
7 rows in set (0.00 sec)

I am trying to run queries extracting from both tables but when I do myslq just hangs and I get no error and no results: This are the queries that are causing problems:

mysql>SELECT logs.src, logs.dst, logs.dstport, src_ipinfo.country, COUNT(1) hits FROM logs, src_ipinfo WHERE logs.src = src_ipinfo.src GROUP BY logs.src, logs.dst, logs.dstport ORDER BY hits DESC;

mysql>SELECT * FROM logs a  , src_ipinfo b WHERE a.src = b.src AND a.dstport= 60595   ORDER BY a.src, a.dst;

The ultimate goal is to search from logs where dstport = 'xxxxx' and the country != 'US' keeping count of how many times that source reached the same dst, dstport combination. Example output:

mysql> describe src_ipinfo;
+----------+--------------+---------+-------+---------+-------+
| src      | dst          | dstport | proto | country | hits  |
+----------+--------------+---------+-------+---------+-------+
| 2.3.45.3 | 10.10.1.23   | 60531   | TCP   | CN      | 3452  |
| 1.2.45.3 | 10.10.1.23   | 80801   | TCP   | NL      | 37    |
| 4.5.45.3 | 10.10.1.23   | 443     | TCP   | IN      | 2     |
+----------+--------------+---------+-------+---------+-------+

I already have individual queries for counting src, dst combo:

mysql>SELECT src, dst, dstport, proto, COUNT(src) hits FROM `logs` WHERE dstport = '60595' GROUP BY src,dst ORDER BY hits DESC;

Now I need to add to that query the country information from src_ipinfo table and further manipulate the result so that I only get non US sources.

bumped to the homepage by Community 19 hours ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

0

Not sure what was happening in the past with my queries but this query worked:

SELECT date,time,logs.src,src_ipinfo.country, dst,proto,sentbyte,rcvdbyte, COUNT(logs.src) hits FROM `logs`, src_ipinfo WHERE dstport = '60595' AND logs.src = src_ipinfo.src GROUP BY logs.src, logs.dst ORDER BY hits DESC;

And the result was:

mysql> SELECT date,time,logs.src,src_ipinfo.country, dst,proto,sentbyte,rcvdbyte, COUNT(logs.src) hits FROM `logs`, src_ipinfo WHERE dstport = '60595' AND logs.src = src_ipinfo.src GROUP BY logs.src, logs.dst ORDER BY hits DESC;
+------------+-----------------+-----------------+---------+----------------+-------+----------+----------+------+
| date       | time            | src             | country | dst            | proto | sentbyte | rcvdbyte | hits |
+------------+-----------------+-----------------+---------+----------------+-------+----------+----------+------+
| 2014-12-29 | 05:41:04.000000 | XXX.XXX.165.3   | NL      | XXX.XXX.20.48  | UDP   |        0 |        0 | 2314 |
| 2014-12-29 | 05:41:04.000000 | XXX.XXX.165.3   | NL      | XXX.XXX.11.131 | UDP   |        0 |        0 | 2301 |
| 2014-12-29 | 08:49:09.000000 | XXX.XXX.85.9    | DZ      | XXX.XXX.20.48  | UDP   |        0 |        0 | 1401 |
| 2014-12-29 | 05:56:44.000000 | XXX.XXX.85.9    | DZ      | XXX.XXX.11.131 | TCP   |        0 |        0 | 1390 |
| 2014-12-29 | 09:02:23.000000 | XXX.XXX.246.99  | CN      | XXX.XXX.20.48  | UDP   |        0 |        0 |  937 |
| 2014-12-29 | 09:02:22.000000 | XXX.XXX.246.99  | CN      | XXX.XXX.11.131 | UDP   |        0 |        0 |  891 |
| 2014-12-29 | 05:45:34.000000 | XXX.XXX.36.164  | BN      | XXX.XXX.20.48  | UDP   |        0 |        0 |  769 |
| 2014-12-29 | 05:45:34.000000 | XXX.XXX.36.164  | BN      | XXX.XXX.11.131 | UDP   |        0 |        0 |  754 |
| 2014-12-29 | 05:36:31.000000 | XXX.XXX.8.218   | CA      | XXX.XXX.11.131 | UDP   |        0 |        0 |  461 |
| 2014-12-29 | 05:36:51.000000 | XXX.XXX.13.21   | CA      | XXX.XXX.20.48  | UDP   |     4168 |   201908 |  403 |
+------------+-----------------+-----------------+---------+----------------+-------+----------+----------+------+

Now I will play with GROUP BY to aggregate countries

0

Those sound like huge tables, are they? If so, you may need some form of Summary Table to gather subtotals each day (or hour), then use another query to generate the 'report' from that table.

PLEASE use SHOW CREATE TABLE; it is more descriptive than DESCRIBE.

WHERE logs.src = src_ipinfo.src

Without any indexes on those fields, the query will take a loooong time.

WHERE a.src = b.src AND a.dstport= 60595
FROM `logs` WHERE dstport = '60595'

Also, logs needs INDEX(dstport)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.