How to flatten results of a table with two related "many" tables?

Question

I've reorganized some tables in my database to be more flexible but I'm not really sure how to write the SQL to extract meaningful data from them.

I have the following tables (somewhat abbreviated for a clearer example):

CREATE TABLE Loans(
    Id int,
    SchemaId int,
    LoanNumber nvarchar(100)
);

CREATE TABLE SchemaFields(
    Id int,
    SchemaId int,
    FieldName nvarchar(255)
);

CREATE TABLE LoanFields(
    Id int,
    LoanId int,
    SchemaFieldId int,
    FieldValue nvarchar(4000)
);

With the following data:

INSERT INTO Loans (Id, SchemaId, LoanNumber) VALUES (1, 1, 'ABC123');

INSERT INTO SchemaFields (Id, SchemaId, FieldName) VALUES (1, 1, 'First Name');
INSERT INTO SchemaFields (Id, SchemaId, FieldName) VALUES (2, 1, 'Last Name');

INSERT INTO LoanFields (Id, LoanId, SchemaFieldId, FieldValue) VALUES (1, 1, 1, 'John');
INSERT INTO LoanFields (Id, LoanId, SchemaFieldId, FieldValue) VALUES (2, 1, 2, 'Doe');

The objective is to get a query that is flat for a loan with all its fields. (In the real world there will likely be between 20-30 fields for the same schema, but we just have 2 in the example):

LoanNumber   First Name    Last Name
----------   -----------   ----------
ABC123       John          Doe

I cannot use a pivot that references the 'First Name' and 'Last Name' because I'll have no idea what will actually be there.

I have a SQL Fiddle here with schema already in place.

How can I get the desired result?

Answer 1

This can be done using the PIVOT function, but since it sounds like you want to change the query based on the schemaId, then you will want to use dynamic SQL.

If you had a known number of values or knew the columns for a specific schemaID, then you could hard-coded the query. A static query would be:

select loannumber,
  [First Name], 
  [Middle Name], 
  [Last Name]
from
(
  select 
    l.loannumber,
    sf.fieldname,
    lf.fieldvalue
  from loans l
  left join loanfields lf
    on l.id = lf.loanid
  left join schemafields sf
    on lf.schemafieldid = sf.id
    and l.schemaid = sf.schemaid
) src
pivot
(
  max(fieldvalue)
  for fieldname in ([First Name], [Middle Name], [Last Name])
)piv;

See SQL Fiddle with Demo.

If you had an unknown number or you want the columns to change based on a SchemaId that you are passing into a procedure, then you will use dynamic SQL to generate the SQL string:

DECLARE @cols AS NVARCHAR(MAX),
    @query  AS NVARCHAR(MAX),
    @schemaId int = 1

select @cols = STUFF((SELECT distinct ',' + QUOTENAME(FieldName) 
                    from SchemaFields 
                    where schemaid = @schemaid
            FOR XML PATH(''), TYPE
            ).value('.', 'NVARCHAR(MAX)') 
        ,1,1,'')

set @query = 'SELECT loannumber,' + @cols + ' 
            from 
            (
              select 
                l.loannumber,
                sf.fieldname,
                lf.fieldvalue
              from loans l
              left join loanfields lf
                on l.id = lf.loanid
              left join schemafields sf
                on lf.schemafieldid = sf.id
                and l.schemaid = sf.schemaid
              where sf.schemaid = '+cast(@schemaid as varchar(10))+'
            ) x
            pivot 
            (
                max(fieldvalue)
                for fieldname in (' + @cols + ')
            ) p '

execute(@query);

See SQL Fiddle with Demo. Both of these queries will generate the result:

| LOANNUMBER | FIRST NAME | LAST NAME | MIDDLE NAME |
-----------------------------------------------------
|     ABC123 |       John |       Doe |      (null) |
|     XYZ789 |    Charles |     Smith |         Lee |

Answer 2

You can use this pattern. For more schema fields, just expand on the 2nd to last line with all the schema field names.

select *
  from (
    select l.LoanNumber, s.FieldName, f.FieldValue
      from Loans l
      join Schemafields s on s.SchemaId = l.SchemaId
      join LoanFields f on f.LoanId = l.Id and f.SchemaFieldId = s.Id) p
pivot (
    max(FieldValue) for FieldName in ([First Name], [Last Name])
    ) v;

If SchemaId = 1 represents the fields for the Loan type, then you can also dynamically generate the full list of schema field names using something like the below.

declare @sql nvarchar(max) =
    stuff((select ',' + quotename(FieldName)
      from SchemaFields
     where SchemaId = 1
       for xml path(''), type).value('/','nvarchar(max)'),1,1,'');
select @sql = '
select *
  from (
    select l.LoanNumber, s.FieldName, f.FieldValue
      from Loans l
      join Schemafields s on s.SchemaId = l.SchemaId
      join LoanFields f on f.LoanId = l.Id and f.SchemaFieldId = s.Id) p
pivot (
    max(FieldValue) for FieldName in (' + @sql + ')
    ) v';
exec (@sql);