To answer your specific questions about this scenario:
if connection B read record A before checkpoint, does it mean it will read the old value?
if connection B read record A after checkpoint, does it mean it will read the new value?
It will read the new, committed value regardless of the
isn't that weird that sometimes connection B get new value and sometimes it get old value, is the way to fix this problem is A needs to manually trigger a checkpoint?
No, issuing a manual
CHECKPOINT is not necessary.
When a query needs to read a particular data page, it first tries to read that page from the buffer pool (RAM). This is called a "logical read."
If that page is not in the buffer pool, it will be read from disk, placed into the buffer pool, and then that in-memory value will be used by the query. This is called a "physical read."
The point being that rows / data pages that have been updated in RAM (so-called "dirty pages") can be read from memory by multiple queries, and by multiple connections, without first needing to be written to the data file on disk.
In fact, a value in the buffer pool can be updated multiple times before being written to disk.
CHECKPOINT process is not really related to isolation levels, locking, or correctness of query results. It's mainly a performance optimization (to prevent lots of small writes from happening all the time) and a recoverability feature (to reduce the time it takes to recover the database to a consistent state from the log).
You can read a lot of in-depth information about these topics in the following Microsoft Docs articles: