First, try to gather accurate statistics:

    ANALYZE a;

If that doesn't help, you can choose the ugly solution of rewriting the query so that you force a nested loop join:

    SELECT count(*)
    FROM a
    CROSS JOIN LATERAL (SELECT FROM b
                        WHERE a.id = b.a_id
                        OFFSET 0)
    WHERE a.num1 IS NOT NULL
      AND a.num2 IS NOT NULL;