Questions tagged [data-cleansing]

Data cleansing is the process of removing or correcting incorrect data from a dataset

Filter by
Sorted by
Tagged with
0 votes
0 answers
30 views

How to filter extraneous Unicode values from a column?

I am cleaning up / priming our PostgreSQL 12 database for future data-related activities (e.g. data encryption). I have tried the following methods to delete non-basic Latin / basic accented Latin / ...
Bona Rae Villarta's user avatar
1 vote
1 answer
274 views

Find duplicate values by joining tables SQL server

Finding the duplicate values in the 'Item_Sales_Detail' table as NULL rows in the 'Sales' and 'Item' tables by joining three tables. 'Sales' table (ID is primary key) ID Invoice Date TotalAmount 10 ...
Mohammad Bastan's user avatar
0 votes
1 answer
51 views

Why get tables cleaned and copied in an SQL-Server DB?

I'm working on an SQL-Server database. Regularly, entries from one table get moved to another one (from entries to Log_Entries) in order not to flood the database. (The Log_Entries get cleaned ...
Dominique's user avatar
  • 521
1 vote
0 answers
27 views

Selecting Duplicates on All Fields

I have an MS Access (no laughing at the back) database I've used to import a bunch of IIS logs into. Having looked at the Excel files I pulled these in from, I'm worried going by the dates that some ...
user788561's user avatar
0 votes
1 answer
2k views

Cleaning up bad data with REGEXP_REPLACE too aggressive with non latin chars

Trying to use REGEXP_REPLACE to clean up a text field in a table. Here's my query: select title original ,REGEXP_REPLACE(title, '[^a-zA-Z0-9]+', ' ') clean from tmp_table2 WHERE txn_title like '%?%' ...
Kenobi's user avatar
  • 143
4 votes
2 answers
14k views

Find rows with similar string values

I have a Microsoft SQL Server 2012 database table with around 7 million crowd-sourced records, primarily containing a string name value with some related details. For nearly every record it seems ...
kscott's user avatar
  • 141
2 votes
1 answer
5k views

How to compare two tables that have no primary key?

So I got two sets of tables at work that I have to compare the data. The fields are identical, but there is no column that has unique entries. (Employee ID, Assignment ID, Employee's last name, ...
vpxoxo's user avatar
  • 21
2 votes
2 answers
63 views

How should I handle measurement errors in a timeseries database?

I have a table used to record measurements sampled at regular intervals on different sensors. Each row records the time, the identifier of the quantity being measured, and the value itself. Now and ...
lindelof's user avatar
  • 225