Questions tagged [google-cloud-sql]
Google Cloud SQL is a service that allows you to create, configure, and use relational databases (currently MySQL and PostgreSQL) that live in Google's cloud. It is a fully-managed service that maintains, manages, and administers your databases, allowing you to focus on your applications and services.
91
questions
2
votes
1
answer
2k
views
How to sync easily between a master database from Google Cloud SQL and a master database from Amazon RDS
I've created a master - slave (replication) in Google Cloud SQL.
I've also created a master - slave (replication) in Amazon RDS.
I want to sync two master MysQL from Google Cloud SQL and Amazon RDS....
CommunityBot
- 1
0
votes
2
answers
1k
views
can't backup postgres 13 with pg_dump
I'm trying to backup a postgresql 13 database with the following command on MacOS:
PGPASSWORD=“my_password” /Library/PostgreSQL/13/bin/pg_dump -h xx.xx.xx.xx -p 5432 -U postgres dbname > my_backup....
CommunityBot
- 1
0
votes
0
answers
23
views
Why can my user even connect?
I need to create a new Postgres user for an application that needs to connect to an existing applications database.
The original database was created using the Cloud SQL dashboard, the new user is ...
- 101
1
vote
1
answer
73
views
Google Cloud SQL MySQL database storage increase due to `tmp_data`. What is tmp_data is Google Cloud MySQL and what might be the cause?
Our Google Cloud MySQL database had a huge and unexpected increase of used space in the span of a couple of days. It went from 15 GB to 16TB! This is also obviously causing issues with our billing as ...
- 18.1k
0
votes
1
answer
326
views
Online schema changes for cloudsql DB
I'm new to cloudsql for mysql. I have a table with trigger. I need to do some ddl changes on that table. However, I can't take any downtime.
What I have tried so far:
I tried with pt-online-schema-...
CommunityBot
- 1
1
vote
2
answers
858
views
can any tweaks improve full table scan time in Postgres/Cloud SQL?
I have a table with 150 million rows in a Postgres database hosted in Cloud SQL (in Google Cloud Platform).
Typically I obviously try to avoid scanning the table, but occasionally I will manually want ...
CommunityBot
- 1
0
votes
1
answer
537
views
MySQL column that auto generates data, longer than UUID
My friend has the problem as shown below. He asked at StackOverFlow and there is no reply yet till now so I wish to help him to try the luck here. Thanks for everyone that helps.
"I would like to ask ...
CommunityBot
- 1
0
votes
0
answers
28
views
How to Efficiently Manage Multiple Databases in a Multi-Tenant Application on GCP
I am building a multi-tenant application using Google Cloud Platform (GCP). My application is deployed on App Engine, and I am using MySQL Cloud SQL for the database. To ensure data isolation, I have ...
2
votes
1
answer
953
views
Postgresql sudden storage increase at regular interval?
I am running Postgresql 11 in production and observing a weird step function kind of thing in storage as follow:
The above graph is for a single day and it is happening everyday at random times. I ...
CommunityBot
- 1
0
votes
0
answers
66
views
Basic setup with Cloud SQL and IAM authentication
Let's say I have a database instance, two databases db1 and db2, and two users user1 and user2@PROJECT_ID.iam (BUILTIN and CLOUD_IAM_SERVICE_ACCOUNT). In other words:
resource "...
- 367
0
votes
0
answers
89
views
invalidating slot "a" because its restart_lsn 6093/6BB6F1B0 exceeds max_slot_wal_keep_size
we use google cloud sql (version is PostgreSQL 15.5), the config as follows:
config, and max_slot_wal_keep_size = -1, but slot always become lost wal status (restart LSN become null) when running for ...
- 40.7k
0
votes
1
answer
124
views
How to get Google Cloud SQL Auth Proxy working on Windows (WSL)?
I am trying to run Google's Cloud SQL Auth Proxy on my Windows 11 machine under WSL (Windows Subsystem for Linux). I downloaded the Cloud SQL Auth Proxy 64-bit executable and run it and get these ...
- 375
1
vote
0
answers
59
views
How to create and manage superusers (like sysadmin) in Google Cloud SQL with SQL Server
I'm a database engineer that works most with SQL Server and with experience with SQL Server and all SQL alternatives in Microsoft Azure and AWS.
Recently I had to work with a managed SQL Server ...
- 3,672
0
votes
1
answer
27
views
Does the client certificate need to be signed by the server certificate?
I am connecting to a GCP Cloud SQL instance using a client certificate issued by the GCP Cloud SQL service. When I created the client key-pair GCP Cloud SQL gave me three files: a server certificate, ...
- 26.6k
2
votes
2
answers
4k
views
PgAdmin 4.14 connection error in cloudSQL : could not send data to server, socket is not connected, could not send ssl negotiation packet
I am trying to connect cloud sql with pgadmin but it throws error when I try to execute query.
"Could not send Data to server, socket is not connected, could not
sent SSL negotiation pocket"
...
- 70.3k
0
votes
1
answer
540
views
How to investigate intermittent Postgres connection errors?
Our webserver (Django-app on Gunicorn running on Google Cloud Run) connects to a Postgres 15 database (on Google Cloud SQL) through Psycopg. Most queries are successful, but recently ~1% of the ...
- 111
0
votes
0
answers
9
views
GCP Point in time recovery log storage to buckets
GCP now offers storing binary logs in Cloud Storage instead of on your MySQL disk, but for older instances, you need to disable and re-enable binary logs to start storing them in Cloud Storage.
My DB ...
0
votes
0
answers
19
views
CloudSQL for Postgres S3 extension
In AWS RDS for Postgres, there is an extension called AWS_S3 that provides functions within Postgres that allow me to import data directly from a bucket into a table and export data from a table ...
- 45
1
vote
1
answer
158
views
What is the best way to migrate a multi-TB MySQL database from on-prem to gcloud Cloud SQL?
I have several multi-TB on-prem MySQL databases I need to migrate into Google Cloud's managed MySQL offering, Cloud SQL. I have migrated two of size ~1TB so far with mysqldump but this method is far ...
- 111
0
votes
2
answers
971
views
Half a million rows deleted still showing same disk size as before in phpMyAdmin
I imported a 800MB SQL file to Google Cloud MySQL.
I have one table that has 200 fields which is amounting to 500MB.
I deleted all except 240 necessary rows in that table.
phpMyAdmin still shows its ...
- 78.9k
1
vote
0
answers
19
views
How to Transfer 100GB database from GCP to AWS
I have a 100GB database hosted in GCP's Cloud SQL offering in us-east1
I'd like to transfer this to AWS lightsail databases and hosted in AP-SOUTHEAST-2.
Within AWS I'm using AWS Lightsail Databases ...
- 111
0
votes
1
answer
275
views
What is the difference between `SHOW OPEN TABLES;` and `SHOW GLOBAL STATUS LIKE 'Open_tables';`?
I need some help understanding a few things.
We use Google Cloud SQL MySQL8.0 and had an incident today where the working theory is that the instance was under memory pressure and took longer than ...
- 78.9k
4
votes
1
answer
7k
views
Set column to randomly chosen value from another table
Consider this:
UPDATE dest
SET dest_col = (SELECT s.src_col FROM source s
WHERE s.name = 'abc'
ORDER BY random() LIMIT 1 OFFSET 0);
My goal: Set dest.dest_col in ...
- 1
6
votes
2
answers
4k
views
Mysql query works in 200ms with 33594 parameters in SELECT IN (xx,xxx,xx) but runs hours with 33595 or more parameters, why?
We had a small malfunction which generated unexpected amount of data for a device. Our code ends up creating a select query along the lines of
SELEXT * from XXX WHERE yyy IN (xxx,xxx,xxx....,xxxx)
...
- 15.5k
0
votes
1
answer
250
views
Google Cloud SQL (MySQL) consume too much CPU during several days
There was noticed that Cloud SQL (MySQL) consumed to much CPU (~100%) during several days.
Firstly, I decided that we have many user online or background jobs. But it was regular workload, nothing ...
2
votes
1
answer
295
views
Why does the page cleaner in MySQL 8 increase the number of active connections to the database?
I have noticed that when the page cleaner is triggered in my MySQL 8 database, it causes a temporary increase in the number of active connections. This can result in increased wait time for new ...
- 184k
0
votes
1
answer
47
views
What is ideal configuration for Mysqld.cnf?
I run a news & media blog with over a few million readers every month. I'm currently using Google Cloud with a 32gb ram server with WordPress.
My server is hitting 84% memory usage due to the ...
- 1,765
2
votes
1
answer
2k
views
Creating manual superuser in cloudsql for postgres
I’m using google managed cloudsql database and I’m trying to create my own superuser manually and I’m getting below message
Note: I’m logged in via existing superuser only(mycurrentuser_dev).
...
CommunityBot
- 1
1
vote
0
answers
102
views
How to recover stopped replication on MySQL?
Several times now my Google Cloud MySQL read replica just stopped replication of the master. Visible symptoms are:
Replication lags rises
on the slave.
The Relay_Log_Pos value of show slave status is ...
- 26.6k
0
votes
0
answers
217
views
How do I update definers to enable MySQL replication?
I am trying to create a MySQL replica in Cloud SQL, Google's managed MySQL service. However I have the following error which won't allow me to start replication:
"kind": "sql#...
- 133
1
vote
1
answer
250
views
Postgres allows to UPDATE however only SELECT is granted
Owner of table x is user dave.
Only SELECT is granted to cloudsqlsuperuser, however user anton is able to update data in table x. How it's possible?
=> \du
...
- 26.6k
0
votes
1
answer
381
views
Enforce table ownership
Here's the deal,: I have postgresql databases (GCP Cloud SQL deployment, I don't care about version) and I want to enforce ownership of tables.
Specifically, an application logs in to the database ...
- 54.9k
1
vote
1
answer
191
views
Google Managed PGSQL DB migration to DigitalOcean Managed PGSQL DB
We are migrating from Google-Cloud PGSQL managed database to a managed PGSQL server on Digital ocean.
The dilemma we are facing is that both Google and Digital ocean are managed so we have no access ...
CommunityBot
- 1
1
vote
1
answer
1k
views
Installing pglogical: Find postgres include folder, add all include folders to C_INCLUDE_PATH
I am trying to install the pglogical extension in my postgres database, which I installed on my macOS with homebrew. I am following instructions provided on the GCP blog as I am preparing to migrate ...
- 111
1
vote
1
answer
2k
views
UPDATE incredibly slow on small PostgreSQL query - EXPLAIN ANALYZE causes it to hang
I'm trying to debug a very slow UPDATE query in one of my tables.
The table name is beepers. It has 2486 rows and weighs 1204 MB.
It has 18 columns and two indices: one on id and the other on ...
- 11
3
votes
2
answers
3k
views
IMPORT FOREIGN SCHEMA fails with Password error for 127.0.0.1 but not for remote IP (of same server)
I have a Postgres server in Google SQL Cloud with multiple databases. I want to create a FOREIGN DATA WRAPPER between them. If I use the server's remote IP it works:
CREATE SERVER "some_db_fdw&...
- 101
1
vote
0
answers
487
views
Query that normally takes 100ms, occasionally takes 30+s. Need help analyzing query plan
I am using Google Cloud SQL. I am using a g1-small. I have a table that looks like this:
+-----------+-------------------+-------------+
| Column | Type | Modifiers |
|-----------+---...
0
votes
1
answer
59
views
Am I missing an index for these slow queries?
I've tried analyzing the EXPLAIN but unsure if I'm missing some helpful index. Queries take 20-30 seconds across 14 million rows.
CREATE TABLE `domains` (
`id` int(10) unsigned NOT NULL ...
- 78.9k
1
vote
0
answers
294
views
How to run pg_xlogdump or pg_waldump in Google CloudSQL? Or use Query Insights to get similar results?
Does anyone know how to look up the transaction that’s associated with an LSN on a CloudSQL Postgres instance?
For example, let’s say I’m trying to find the transaction associated with LSN 842/...
- 141
1
vote
0
answers
122
views
X-devAPI not working inside of Google cloud SQL
I am tryng to use X-devAPI inside of Google cloud SQL but getting the following error from MySQL client
comp@user:~$ sudo mysqlsh -u root -h XX.XXX.XX.XXX --mysql --dba enableXProtocol
[sudo] ...
0
votes
0
answers
709
views
Revoke CREATE for a user from Google Cloud SQL Postgres?
I'm using Google Cloud SQL Postgres and having difficulty revoking privileges to create tables from a user.
I first removed all roles and attributes from the user (truncated for brevity):
mydatabase=&...
- 101
0
votes
0
answers
2k
views
faster way to update very large table (150 million rows) in Postgres
I have a very large table (about 150 million rows) in Postgres 12. It has a simple structure:
col1 (VARCHAR)
col2 (VARCHAR)
col3 (VARCHAR)
col4 (VARCHAR)
The PK is (col1, col2, col3, col4), so as a ...
- 213
0
votes
1
answer
155
views
GCP Cloudsql Mysql Replica unresponsive after mass delete on Master
Directed here from S/O
We have a Master/Replica configuration for Mysql innodb(5.7.32) databases in Cloud SQL... We have a single Table (let's call it Master table) partitioned on Two keys having both ...
- 78.9k
0
votes
1
answer
32
views
Merging one SQL database into another
Currently in my CI I'm completely wiping production db with development db. That was fine for testing but I'm coming to a stage where I need to start merging instead of completely wiping.
Right now ...
- 78.9k
1
vote
0
answers
152
views
GCP Cloudsql Mysql Replica unresponsive after mass delete on Master
We have a Master/Replica configuration for Mysql innodb(5.7.32) databases in Cloud SQL... We have a single Table (let's call it Master table) partitioned on Two keys having both primary and non-...
- 11
1
vote
1
answer
803
views
How do I see where postgres privileges are coming from for a role?
Im using google cloudsql but this is a more generic postgres question.
I have a role (that is already created and managed by cloudsql) called cloudsqliamserviceaccount.
The cloudsqliamserviceaccount ...
- 54.9k
0
votes
0
answers
231
views
Tablespace in cloudsql for mysql
I'm new to mysql and GC. It may sound like opinion based question, so sorry for that. But, I really need a right opinion. My company wants to move their on-prem mysql to GC managed mysql. I was ...
- 362
0
votes
0
answers
96
views
MySQL8 instance crashes on CloudSQL
I had a MySQL 5.7 instance on Google CloudSQL. Now i created a MySQL 8 instance.
The configuration is pretty much the same (except i use 2 CPUs instead of one, 3.75GB of RAM). Now the default config ...
- 101
1
vote
1
answer
68
views
Constant writing operations on google cloud database
I have a MySQL 8.0 cluster database and so far I'm using it to test a couple of things, so I only write from time to time. Problem is that I have realised that there are write operations all the time. ...
- 13
3
votes
1
answer
418
views
Half of the partitions are empty on a KEY partition
We are trying to move data from one table that currently has over 1B rows to a partitioned table.
But as we started to do so we notice that half of the partitions are empty, why is that?
Table is ...
- 15.5k