Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

I can find what DLL supports English word breakers by using sp_help_fulltext_system_components but I have not been able to find an actual list of the word breaking characters for English (like blank, ., %, etc.).

Anyone know of a source for this info?

share|improve this question

2 Answers 2

SQL Server (any version) considers any non alphanumeric character as a potential word breaker. Than means that there are situations where characters expected to break words are handled in a different way (direct or expanded) by the server.

A typical word breaker is the dot ('.')

These are some variations of dot handling (AICI).

select display_term from sys.dm_fts_parser('"The great .NET."', 1033, 0, 0);

returns "the", "great" and ".net"

select display_term from sys.dm_fts_parser('"The great ASP.NET."', 1033, 0, 0);

returns "the", "great", "asp.net", "asp" and "net"

select display_term from sys.dm_fts_parser('"G.I.S."', 1033, 0, 0);

returns "g.i.s." and "gis"

select display_term from sys.dm_fts_parser('"3.14"', 1033, 0, 0);

returns "3.14" and "nn3d14"

And it's not only the dot.

select display_term from sys.dm_fts_parser('"data-driven apps"', 1033, 0, 0);

returns "data-driven", "data", "driven" and "apps"

select display_term from sys.dm_fts_parser('"1-0"', 1033, 0, 0);

returns "1-0", "1", "nn1", "0" and "nn0"

select display_term from sys.dm_fts_parser('"c# j# f# a#"', 1033, 0, 0);

returns "c#", "j#", "f" and "a" (f# is missing as a rule)

select display_term, * from sys.dm_fts_parser('"c c+ c++"', 1033, 0, 0);

returns "c", "c" and "c++"

The list of the (potential) word breakers (Stuart shows the way to get it in his answer) is the easy part. The difficult part is to get the list of the internal rules in use; they're handled/included in MSWB7*.dll, NaturalLanguage6.dll and NL7*.dll located at DRIVE:\Program Files\Microsoft SQL Server\INSTANCENAME\MSSQL\Binn

share|improve this answer

This isn't an official list, but using a loop to work through a list of characters, and using sys.dm_fts_parser like so:


declare @i integer
declare @cnt integer
set @i=0
while @i<255
begin
  set @cnt=0
  select @cnt=COUNT(1) FROM sys.dm_fts_parser ('"word1'+CHAR(@i)+'word2"', 1033, 0, 0)
  if @cnt>1
    begin
      print 'this char - '+char(@i)+' - char('+convert(varchar(3),@i)+') is a word breaker'
    end
  set @i=@i+1
end 

I can generate a list of characters that sys.dm_fts_parser reckons break the words. (sys.dm_fts_parser returns a row for every 'word' found in the import, so if it returns more than 1 row we had a word breaker)

This could be expanded to extended/non-english character sets by using nchar() rather than char() (and a bigger value for @i), and changing parameter 2 (lcid) in the call to sys.dm_fts_parser

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.