The tag has no usage guidance.

learn more… | top users | synonyms

1
vote
1answer
29 views

(SOLVED) Mysql bug decimal comparison

I'm currently writing unit tests for a stored procedure that calculates the amount of taxes and the tax rate based on the country/province the customer lives in. Somehow, MySQL considers that "14.975"...
3
votes
1answer
116 views

How are precision and scale calculated when decimals are multiplied?

When decimals are divided, the precision and the scale of the result value are calculated due to this formula: max precision = (p1 - s1 + s2) + MAX(6, s1 + p2 + 1) -- up to 38 max scale = MAX(...
3
votes
2answers
5k views

Convert string numeric values with comma as decimal separator to NUMERIC(10, 2)

I have an SQL table of varchar colums which contain Greek formated numbers (. as thousand seperator and comma as decimal seperator) The classic conversion CONVERT(numeric(10,2),REPLACE([value],',','....
0
votes
2answers
61 views

Mysql Data truncation for Valid DECIMAL

create table test(id INT, value DECIMAL(25,6)); insert into test values (16, 10e16); insert into test values (17, 10e17); -- Works till here insert into test values (18, 10e18); -- Fails here. Data ...
0
votes
0answers
54 views

MySQL - Get actual precision and scale of number

I would like to query the integer part and fractional part of all decimal(8,4) values in a column. By that I mean, I want to know the actual given values, and not the max allowed by the data type, ...
0
votes
1answer
45 views

Calculations involving thresholds between two tables

I have a table comment and a table price like below. The key columns in both tables are ticker_id, price_datetime and price_open. The last column threshold in table comment (which currently NULL) is ...
1
vote
0answers
82 views

Migration error: “Invalid date literal detected” when reading decimal

I'm attempting a migration from MSSQL to MySQL and keep coming across a "Invalid date literal detected" error when reading certain decimal columns. I'm running the following (all up-to-date as of ...
7
votes
4answers
7k views

How does SQL Server determine precision / scale?

I'm running SQL Server 2012 SELECT 0.15 * 30 / 360, 0.15 / 360 * 30 Results: 0.012500, 0.012480 This one is even mor confusing to me: DECLARE @N INT = 360 DECLARE @I DECIMAL(38,26)...
3
votes
1answer
186 views

Why doesn't `1` fit into a decimal(4, 4) column?

I have a decimal(4, 4) column in MS SQL Server 2008 R2. If I understand correctly, that means: Precision of 4, ie up to four digits can be stored after the decimal place Scale of 4, ie a a total of ...
8
votes
2answers
17k views

Automatic decimal rounding issue

The question is relatively simple. I need to calculate 3 columns where the mid results are huge decimals, and I'm running into a problem early on with SQL Server basically rounding the decimals ...