From a purely relational point of view (prior to sixth normal form), I don't see any need to move a set of columns out into a separate table, just because they are frequently null.
As a trivial example, consider a customer account table with an end date as one of the columns - until the customer closes their account, the end date will be NULL. You are therefore likely to have a large number of NULL values in the end date column, yet it would be a bad choice to move this out into a separate table.
However, as an implementation issue, there may be good reasons to split out certain fields into separate tables. For example, consider a customer address table; certain addresses may require complicated delivery instructions, yet the vast majority are not likely to require them. In such circumstances, it would make sense to have a separate table for address delivery instructions, even if it has the same key (customer address ID) as the main customer address table.
(As a side note, some experts on relational theory - such as Chris Date and Fabian Pascal - are opposed to allowing NULLs within relational databases at all.)