Bumped by Community user
    Bumped by Community user
    Bumped by Community user
6 added 107 characters in body
source | link
mike@mike-thinks:~/Data/on_2018_04_25_16_43_17$ ./importCSVFiles
{"error":{"root_cause":
            [{"type":"mapper_parsing_exception","reason":"failed to parse"}],
          "type":"mapper_parsing_exception",
          "reason":"failed to parse",
          "caused_by":{"type":"i_o_exception","reason":"Illegal unquoted character ((CTRL-CHAR, code 13)): 
              has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@e18584; line: 1, column: 88]"}
         },"status":400 
}
mike@mike-thinks:~/Data/on_2018_04_25_16_43_17$ ./importCSVFiles
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"i_o_exception","reason":"Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@e18584; line: 1, column: 88]"}},"status":400}
mike@mike-thinks:~/Data/on_2018_04_25_16_43_17$ ./importCSVFiles
{"error":{"root_cause":
            [{"type":"mapper_parsing_exception","reason":"failed to parse"}],
          "type":"mapper_parsing_exception",
          "reason":"failed to parse",
          "caused_by":{"type":"i_o_exception","reason":"Illegal unquoted character ((CTRL-CHAR, code 13)): 
              has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@e18584; line: 1, column: 88]"}
         },"status":400 
}
5 added 38 characters in body
source | link

I have several csv files on university courses that all seem linked by an ID, that you can find here, and I wondered how to put them on Elasticsearch. I know, thanks to this video and Logstash, how to insert one sole file csv file to Elasticsearch. But do you know how to insert several such as those in the provided link ?

At the moment I started with a first .config file for a first csv file : ACCREDITATION.csv. But it would be painful to write them all...

The .config file is :

input{
    file{
        path =>"Users/mike/Data/ACCREDITATION.csv"
        start_position => "begining"
        sincedb_path => "/dev/null"
    }
}

filter{
    csv{
        separator => ","
        columns => ['PUBUKPRN', 'UKPRN', 'KISCOURSEID', 'KISMODE', 'ACCTYPE', 'ACCDEPEND', 'ACCDEPENDURL', 'ACCDEPENDURLW']

    }
    mutate{convert => ["PUBUKPRN","integer"]}
    mutate{convert => ["UKPRN","integer"]}
    mutate{convert => ["KISMODE","integer"]}
    mutate{convert => ["ACCTYPE","integer"]}
    mutate{convert => ["ACCDEPEND","integer"]}
}

output{
    elasticsearch{
        hosts =>"localhost"
        index =>"accreditation"
        document_type =>"accreditaiton keys"
    }
    stdout{}
}

Update May, 3rd

Without knowing how to use a .config file to implement csv files to Elasticsearch, I fell back to Elastic blog and tried to do a shell script importSVFiles for a first .csv file before trying to generalize the approach :

importCSVFiles :

#!/bin/bash
while read f1
do        
   curl -XPOST 'https://XXX.us-east-1.aws.found.io:9243/courses/accreditation' -H "Content-Type: application/json" -u elastic:XXX -d "{ \"accreditation\": \"$f1\" }"
done < AccreditationByHep.csv

Yet I received a mapper_parsing_exception on the terminal :

mike@mike-thinks:~/Data/on_2018_04_25_16_43_17$ ./importCSVFiles
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"i_o_exception","reason":"Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@e18584; line: 1, column: 88]"}},"status":400}

I have several csv files on university courses that all seem linked by an ID, that you can find here, and I wondered how to put them on Elasticsearch. I know, thanks to this video and Logstash, how to insert one sole file csv file to Elasticsearch. But do you know how to insert several such as those in the provided link ?

At the moment I started with a first .config file for a first csv file : ACCREDITATION.csv. But it would be painful to write them all...

The .config file is :

input{
    file{
        path =>"Users/mike/Data/ACCREDITATION.csv"
        start_position => "begining"
        sincedb_path => "/dev/null"
    }
}

filter{
    csv{
        separator => ","
        columns => ['PUBUKPRN', 'UKPRN', 'KISCOURSEID', 'KISMODE', 'ACCTYPE', 'ACCDEPEND', 'ACCDEPENDURL', 'ACCDEPENDURLW']

    }
    mutate{convert => ["PUBUKPRN","integer"]}
    mutate{convert => ["UKPRN","integer"]}
    mutate{convert => ["KISMODE","integer"]}
    mutate{convert => ["ACCTYPE","integer"]}
    mutate{convert => ["ACCDEPEND","integer"]}
}

output{
    elasticsearch{
        hosts =>"localhost"
        index =>"accreditation"
        document_type =>"accreditaiton keys"
    }
    stdout{}
}

Update May, 3rd

Without knowing how to use a .config file to implement csv files to Elasticsearch, I fell back to Elastic blog and tried to do a shell script for a first .csv file before trying to generalize the approach :

#!/bin/bash
while read f1
do        
   curl -XPOST 'https://XXX.us-east-1.aws.found.io:9243/courses/accreditation' -H "Content-Type: application/json" -u elastic:XXX -d "{ \"accreditation\": \"$f1\" }"
done < AccreditationByHep.csv

Yet I received a mapper_parsing_exception

mike@mike-thinks:~/Data/on_2018_04_25_16_43_17$ ./importCSVFiles
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"i_o_exception","reason":"Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@e18584; line: 1, column: 88]"}},"status":400}

I have several csv files on university courses that all seem linked by an ID, that you can find here, and I wondered how to put them on Elasticsearch. I know, thanks to this video and Logstash, how to insert one sole file csv file to Elasticsearch. But do you know how to insert several such as those in the provided link ?

At the moment I started with a first .config file for a first csv file : ACCREDITATION.csv. But it would be painful to write them all...

The .config file is :

input{
    file{
        path =>"Users/mike/Data/ACCREDITATION.csv"
        start_position => "begining"
        sincedb_path => "/dev/null"
    }
}

filter{
    csv{
        separator => ","
        columns => ['PUBUKPRN', 'UKPRN', 'KISCOURSEID', 'KISMODE', 'ACCTYPE', 'ACCDEPEND', 'ACCDEPENDURL', 'ACCDEPENDURLW']

    }
    mutate{convert => ["PUBUKPRN","integer"]}
    mutate{convert => ["UKPRN","integer"]}
    mutate{convert => ["KISMODE","integer"]}
    mutate{convert => ["ACCTYPE","integer"]}
    mutate{convert => ["ACCDEPEND","integer"]}
}

output{
    elasticsearch{
        hosts =>"localhost"
        index =>"accreditation"
        document_type =>"accreditaiton keys"
    }
    stdout{}
}

Update May, 3rd

Without knowing how to use a .config file to implement csv files to Elasticsearch, I fell back to Elastic blog and tried to do a shell script importSVFiles for a first .csv file before trying to generalize the approach :

importCSVFiles :

#!/bin/bash
while read f1
do        
   curl -XPOST 'https://XXX.us-east-1.aws.found.io:9243/courses/accreditation' -H "Content-Type: application/json" -u elastic:XXX -d "{ \"accreditation\": \"$f1\" }"
done < AccreditationByHep.csv

Yet I received a mapper_parsing_exception on the terminal :

mike@mike-thinks:~/Data/on_2018_04_25_16_43_17$ ./importCSVFiles
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"i_o_exception","reason":"Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@e18584; line: 1, column: 88]"}},"status":400}
4 added 1071 characters in body
source | link

I have several csv files on university courses that all seem linked by an ID, that you can find here, and I wondered how to put them on Elasticsearch. I know, thanks to this video and Logstash, how to insert one sole file csv file to Elasticsearch. But do you know how to insert several such as those in the provided link ?

At the moment I started with a first .config file for a first csv file : ACCREDITATION.csv. But it would be painful to write them all...

The .config file is :

input{
    file{
        path =>"Users/mike/Data/ACCREDITATION.csv"
        start_position => "begining"
        sincedb_path => "/dev/null"
    }
}

filter{
    csv{
        separator => ","
        columns => ['PUBUKPRN', 'UKPRN', 'KISCOURSEID', 'KISMODE', 'ACCTYPE', 'ACCDEPEND', 'ACCDEPENDURL', 'ACCDEPENDURLW']

    }
    mutate{convert => ["PUBUKPRN","integer"]}
    mutate{convert => ["UKPRN","integer"]}
    mutate{convert => ["KISMODE","integer"]}
    mutate{convert => ["ACCTYPE","integer"]}
    mutate{convert => ["ACCDEPEND","integer"]}
}

output{
    elasticsearch{
        hosts =>"localhost"
        index =>"accreditation"
        document_type =>"accreditaiton keys"
    }
    stdout{}
}

Update May, 3rd

Without knowing how to use a .config file to implement csv files to Elasticsearch, I fell back to Elastic blog and tried to do a shell script for a first .csv file before trying to generalize the approach :

#!/bin/bash
while read f1
do        
   curl -XPOST 'https://XXX.us-east-1.aws.found.io:9243/courses/accreditation' -H "Content-Type: application/json" -u elastic:XXX -d "{ \"accreditation\": \"$f1\" }"
done < AccreditationByHep.csv

Yet I received a mapper_parsing_exception

mike@mike-thinks:~/Data/on_2018_04_25_16_43_17$ ./importCSVFiles
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"i_o_exception","reason":"Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@e18584; line: 1, column: 88]"}},"status":400}

I have several csv files on university courses that all seem linked by an ID, that you can find here, and I wondered how to put them on Elasticsearch. I know, thanks to this video and Logstash, how to insert one sole file csv file to Elasticsearch. But do you know how to insert several such as those in the provided link ?

At the moment I started with a first .config file for a first csv file : ACCREDITATION.csv. But it would be painful to write them all...

The .config file is :

input{
    file{
        path =>"Users/mike/Data/ACCREDITATION.csv"
        start_position => "begining"
        sincedb_path => "/dev/null"
    }
}

filter{
    csv{
        separator => ","
        columns => ['PUBUKPRN', 'UKPRN', 'KISCOURSEID', 'KISMODE', 'ACCTYPE', 'ACCDEPEND', 'ACCDEPENDURL', 'ACCDEPENDURLW']

    }
    mutate{convert => ["PUBUKPRN","integer"]}
    mutate{convert => ["UKPRN","integer"]}
    mutate{convert => ["KISMODE","integer"]}
    mutate{convert => ["ACCTYPE","integer"]}
    mutate{convert => ["ACCDEPEND","integer"]}
}

output{
    elasticsearch{
        hosts =>"localhost"
        index =>"accreditation"
        document_type =>"accreditaiton keys"
    }
    stdout{}
}

I have several csv files on university courses that all seem linked by an ID, that you can find here, and I wondered how to put them on Elasticsearch. I know, thanks to this video and Logstash, how to insert one sole file csv file to Elasticsearch. But do you know how to insert several such as those in the provided link ?

At the moment I started with a first .config file for a first csv file : ACCREDITATION.csv. But it would be painful to write them all...

The .config file is :

input{
    file{
        path =>"Users/mike/Data/ACCREDITATION.csv"
        start_position => "begining"
        sincedb_path => "/dev/null"
    }
}

filter{
    csv{
        separator => ","
        columns => ['PUBUKPRN', 'UKPRN', 'KISCOURSEID', 'KISMODE', 'ACCTYPE', 'ACCDEPEND', 'ACCDEPENDURL', 'ACCDEPENDURLW']

    }
    mutate{convert => ["PUBUKPRN","integer"]}
    mutate{convert => ["UKPRN","integer"]}
    mutate{convert => ["KISMODE","integer"]}
    mutate{convert => ["ACCTYPE","integer"]}
    mutate{convert => ["ACCDEPEND","integer"]}
}

output{
    elasticsearch{
        hosts =>"localhost"
        index =>"accreditation"
        document_type =>"accreditaiton keys"
    }
    stdout{}
}

Update May, 3rd

Without knowing how to use a .config file to implement csv files to Elasticsearch, I fell back to Elastic blog and tried to do a shell script for a first .csv file before trying to generalize the approach :

#!/bin/bash
while read f1
do        
   curl -XPOST 'https://XXX.us-east-1.aws.found.io:9243/courses/accreditation' -H "Content-Type: application/json" -u elastic:XXX -d "{ \"accreditation\": \"$f1\" }"
done < AccreditationByHep.csv

Yet I received a mapper_parsing_exception

mike@mike-thinks:~/Data/on_2018_04_25_16_43_17$ ./importCSVFiles
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"i_o_exception","reason":"Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@e18584; line: 1, column: 88]"}},"status":400}
3 added 796 characters in body
source | link
2 Precised title and question
source | link
1
source | link