Questions tagged [google-cloud-sql]

Google Cloud SQL is a service that allows you to create, configure, and use relational databases (currently MySQL and PostgreSQL) that live in Google's cloud. It is a fully-managed service that maintains, manages, and administers your databases, allowing you to focus on your applications and services.

Filter by
Sorted by
Tagged with
0 votes
0 answers
64 views

How to investigate intermittent Postgres connection errors?

Our webserver (Django-app on Gunicorn running on Google Cloud Run) connects to a Postgres 15 database (on Google Cloud SQL) through Psycopg. Most queries are successful, but recently ~1% of the ...
Koen De Wit's user avatar
0 votes
2 answers
101 views

Half a million rows deleted still showing same disk size as before in phpMyAdmin

I imported a 800MB SQL file to Google Cloud MySQL. I have one table that has 200 fields which is amounting to 500MB. I deleted all except 240 necessary rows in that table. phpMyAdmin still shows its ...
anjanesh's user avatar
  • 269
1 vote
0 answers
18 views

How to Transfer 100GB database from GCP to AWS

I have a 100GB database hosted in GCP's Cloud SQL offering in us-east1 I'd like to transfer this to AWS lightsail databases and hosted in AP-SOUTHEAST-2. Within AWS I'm using AWS Lightsail Databases ...
Peter Kneale's user avatar
0 votes
1 answer
123 views

What is the difference between `SHOW OPEN TABLES;` and `SHOW GLOBAL STATUS LIKE 'Open_tables';`?

I need some help understanding a few things. We use Google Cloud SQL MySQL8.0 and had an incident today where the working theory is that the instance was under memory pressure and took longer than ...
Moritz Schmitz v. Hülst's user avatar
0 votes
0 answers
37 views

Connect SnapLogic to Google Cloud SQL Postgres database

I have a SnapLogic pipeline that returns tabular data. I'm trying to write the data to a Cloud SQL postgres instance. However, the Postgres Snap only provides space for a username/password. I've ...
Matt's user avatar
  • 271
6 votes
2 answers
4k views

Mysql query works in 200ms with 33594 parameters in SELECT IN (xx,xxx,xx) but runs hours with 33595 or more parameters, why?

We had a small malfunction which generated unexpected amount of data for a device. Our code ends up creating a select query along the lines of SELEXT * from XXX WHERE yyy IN (xxx,xxx,xxx....,xxxx) ...
Tuxie's user avatar
  • 182
0 votes
1 answer
206 views

Google Cloud SQL (MySQL) consume too much CPU during several days

There was noticed that Cloud SQL (MySQL) consumed to much CPU (~100%) during several days. Firstly, I decided that we have many user online or background jobs. But it was regular workload, nothing ...
Anton Komarov's user avatar
2 votes
1 answer
218 views

Why does the page cleaner in MySQL 8 increase the number of active connections to the database?

I have noticed that when the page cleaner is triggered in my MySQL 8 database, it causes a temporary increase in the number of active connections. This can result in increased wait time for new ...
user avatar
0 votes
1 answer
40 views

What is ideal configuration for Mysqld.cnf?

I run a news & media blog with over a few million readers every month. I'm currently using Google Cloud with a 32gb ram server with WordPress. My server is hitting 84% memory usage due to the ...
Jaya's user avatar
  • 11
1 vote
0 answers
94 views

How to recover stopped replication on MySQL?

Several times now my Google Cloud MySQL read replica just stopped replication of the master. Visible symptoms are: Replication lags rises on the slave. The Relay_Log_Pos value of show slave status is ...
Laures's user avatar
  • 111
1 vote
1 answer
136 views

What is the best way to migrate a multi-TB MySQL database from on-prem to gcloud Cloud SQL?

I have several multi-TB on-prem MySQL databases I need to migrate into Google Cloud's managed MySQL offering, Cloud SQL. I have migrated two of size ~1TB so far with mysqldump but this method is far ...
CClarke's user avatar
  • 133
0 votes
0 answers
154 views

How do I update definers to enable MySQL replication?

I am trying to create a MySQL replica in Cloud SQL, Google's managed MySQL service. However I have the following error which won't allow me to start replication: "kind": "sql#...
CClarke's user avatar
  • 133
1 vote
1 answer
237 views

Postgres allows to UPDATE however only SELECT is granted

Owner of table x is user dave. Only SELECT is granted to cloudsqlsuperuser, however user anton is able to update data in table x. How it's possible? => \du ...
TohaSt's user avatar
  • 13
0 votes
1 answer
278 views

Enforce table ownership

Here's the deal,: I have postgresql databases (GCP Cloud SQL deployment, I don't care about version) and I want to enforce ownership of tables. Specifically, an application logs in to the database ...
PeV's user avatar
  • 103
1 vote
1 answer
1k views

Installing pglogical: Find postgres include folder, add all include folders to C_INCLUDE_PATH

I am trying to install the pglogical extension in my postgres database, which I installed on my macOS with homebrew. I am following instructions provided on the GCP blog as I am preparing to migrate ...
Dinosaur Reporter's user avatar
1 vote
1 answer
1k views

UPDATE incredibly slow on small PostgreSQL query - EXPLAIN ANALYZE causes it to hang

I'm trying to debug a very slow UPDATE query in one of my tables. The table name is beepers. It has 2486 rows and weighs 1204 MB. It has 18 columns and two indices: one on id and the other on ...
R. Gosman's user avatar
1 vote
0 answers
406 views

Query that normally takes 100ms, occasionally takes 30+s. Need help analyzing query plan

I am using Google Cloud SQL. I am using a g1-small. I have a table that looks like this: +-----------+-------------------+-------------+ | Column | Type | Modifiers | |-----------+---...
Dominic Bou-Samra's user avatar
0 votes
1 answer
58 views

Am I missing an index for these slow queries?

I've tried analyzing the EXPLAIN but unsure if I'm missing some helpful index. Queries take 20-30 seconds across 14 million rows. CREATE TABLE `domains` ( `id` int(10) unsigned NOT NULL ...
d-_-b's user avatar
  • 131
2 votes
1 answer
2k views

Creating manual superuser in cloudsql for postgres

I’m using google managed cloudsql database and I’m trying to create my own superuser manually and I’m getting below message Note: I’m logged in via existing superuser only(mycurrentuser_dev). ...
channa's user avatar
  • 350
1 vote
0 answers
258 views

How to run pg_xlogdump or pg_waldump in Google CloudSQL? Or use Query Insights to get similar results?

Does anyone know how to look up the transaction that’s associated with an LSN on a CloudSQL Postgres instance? For example, let’s say I’m trying to find the transaction associated with LSN 842/...
modulitos's user avatar
  • 141
0 votes
2 answers
871 views

can't backup postgres 13 with pg_dump

I'm trying to backup a postgresql 13 database with the following command on MacOS: PGPASSWORD=“my_password” /Library/PostgreSQL/13/bin/pg_dump -h xx.xx.xx.xx -p 5432 -U postgres dbname > my_backup....
Sam Leurs's user avatar
  • 141
0 votes
0 answers
615 views

Revoke CREATE for a user from Google Cloud SQL Postgres?

I'm using Google Cloud SQL Postgres and having difficulty revoking privileges to create tables from a user. I first removed all roles and attributes from the user (truncated for brevity): mydatabase=&...
ashgromnies's user avatar
0 votes
0 answers
2k views

faster way to update very large table (150 million rows) in Postgres

I have a very large table (about 150 million rows) in Postgres 12. It has a simple structure: col1 (VARCHAR) col2 (VARCHAR) col3 (VARCHAR) col4 (VARCHAR) The PK is (col1, col2, col3, col4), so as a ...
Stephen's user avatar
  • 213
0 votes
1 answer
273 views

Online schema changes for cloudsql DB

I'm new to cloudsql for mysql. I have a table with trigger. I need to do some ddl changes on that table. However, I can't take any downtime. What I have tried so far: I tried with pt-online-schema-...
channa's user avatar
  • 350
0 votes
1 answer
31 views

Merging one SQL database into another

Currently in my CI I'm completely wiping production db with development db. That was fine for testing but I'm coming to a stage where I need to start merging instead of completely wiping. Right now ...
user avatar
0 votes
1 answer
143 views

GCP Cloudsql Mysql Replica unresponsive after mass delete on Master

Directed here from S/O We have a Master/Replica configuration for Mysql innodb(5.7.32) databases in Cloud SQL... We have a single Table (let's call it Master table) partitioned on Two keys having both ...
Faraz Beg's user avatar
1 vote
0 answers
142 views

GCP Cloudsql Mysql Replica unresponsive after mass delete on Master

We have a Master/Replica configuration for Mysql innodb(5.7.32) databases in Cloud SQL... We have a single Table (let's call it Master table) partitioned on Two keys having both primary and non-...
Faraz Beg's user avatar
1 vote
1 answer
679 views

How do I see where postgres privileges are coming from for a role?

Im using google cloudsql but this is a more generic postgres question. I have a role (that is already created and managed by cloudsql) called cloudsqliamserviceaccount. The cloudsqliamserviceaccount ...
user1028270's user avatar
0 votes
0 answers
210 views

Tablespace in cloudsql for mysql

I'm new to mysql and GC. It may sound like opinion based question, so sorry for that. But, I really need a right opinion. My company wants to move their on-prem mysql to GC managed mysql. I was ...
channa's user avatar
  • 350
0 votes
0 answers
92 views

MySQL8 instance crashes on CloudSQL

I had a MySQL 5.7 instance on Google CloudSQL. Now i created a MySQL 8 instance. The configuration is pretty much the same (except i use 2 CPUs instead of one, 3.75GB of RAM). Now the default config ...
fredalex's user avatar
  • 101
3 votes
2 answers
2k views

IMPORT FOREIGN SCHEMA fails with Password error for 127.0.0.1 but not for remote IP (of same server)

I have a Postgres server in Google SQL Cloud with multiple databases. I want to create a FOREIGN DATA WRAPPER between them. If I use the server's remote IP it works: CREATE SERVER "some_db_fdw&...
WillyC's user avatar
  • 133
1 vote
1 answer
61 views

Constant writing operations on google cloud database

I have a MySQL 8.0 cluster database and so far I'm using it to test a couple of things, so I only write from time to time. Problem is that I have realised that there are write operations all the time. ...
Juan Wilde's user avatar
3 votes
1 answer
387 views

Half of the partitions are empty on a KEY partition

We are trying to move data from one table that currently has over 1B rows to a partitioned table. But as we started to do so we notice that half of the partitions are empty, why is that? Table is ...
ffox003's user avatar
  • 185
0 votes
0 answers
227 views

Google cloud MySQL replication with partition

I recently altered a table in MySQL 8.0 to with new Primary key and created partitions with them to improve the query performance. which worked fine in main instance. My assumption was that the ...
Azar's user avatar
  • 1
1 vote
1 answer
569 views

download mysql slow query log on google cloud sql

I'm having a difficult time locating the log file to download. When running show global variables like '%slow%'; I can see it is located /mysql/logs/slow.log When I log into the instance though ...
Goddard's user avatar
  • 111
1 vote
0 answers
560 views

Keeping the same order of selected records while inserting to another table

I am running a below query in big query insert into `test-project.temp_test_dataset.cluster_records` (pm,col1,col2,col3,total) select pm,col1,col2,col3,total from ( select 'ingest' as pm,col1,col2,...
vikrant rana's user avatar
0 votes
0 answers
198 views

how to enable CDC in google cloud sql server for AWS DMS replication?

I am trying to migrate data from GCP cloud sql server to AWS aurora MySQL using DMS CDC, for this I need to enable CDC on DMS source database which cloud SQL. As per the AWS documentation I need to ...
Pand005's user avatar
  • 151
6 votes
1 answer
8k views

ERROR: role "user1" cannot be dropped because some objects depend on it

On my Google Cloud SQL database instance (Postgres 9.6), I am unable to drop a user/role via the GCP Console that was created via the GCP Console. When I try, the Console shows a notification that ...
Joe Hawkins's user avatar
0 votes
0 answers
50 views

How to get disk statistics in the Postgres' Cloud SQL?

We are having performance issues in our database probably connected to the disk performance (maximum read MB/s). Therefore, I want to see some statistics of the disks. However, there is no such ...
sacherus's user avatar
  • 101
1 vote
2 answers
642 views

can any tweaks improve full table scan time in Postgres/Cloud SQL?

I have a table with 150 million rows in a Postgres database hosted in Cloud SQL (in Google Cloud Platform). Typically I obviously try to avoid scanning the table, but occasionally I will manually want ...
Stephen's user avatar
  • 213
1 vote
0 answers
129 views

Restore Stored Procedure from Google Cloud MYSQL?

I accidentally overwrote a MYSQL stored procedure through the Workbench client connected to a GCP MYSQL instance. I tried restoring a previous backup to recover it, but that only restored the tables - ...
Baz's user avatar
  • 111
0 votes
0 answers
339 views

Choosing right database for storing bank transactions

I am starting a new project within GCP and I am trying to choose a right tool for storing bank transactions: I don't need transactions, these will be basically write-only, no updates I don't need ...
Vojtěch's user avatar
1 vote
0 answers
37 views

Move Our Google Cloud SQL Instance to Compute Engine

We are running several Cloud SQL Instances on GCP and plan to move all of them to some Compute Engine instances, depending on the applications that use it. Our main concern is cost of the server. We ...
Prabowo Murti's user avatar
2 votes
1 answer
785 views

Postgresql sudden storage increase at regular interval?

I am running Postgresql 11 in production and observing a weird step function kind of thing in storage as follow: The above graph is for a single day and it is happening everyday at random times. I ...
hardik24's user avatar
1 vote
1 answer
169 views

Google Managed PGSQL DB migration to DigitalOcean Managed PGSQL DB

We are migrating from Google-Cloud PGSQL managed database to a managed PGSQL server on Digital ocean. The dilemma we are facing is that both Google and Digital ocean are managed so we have no access ...
sqwale's user avatar
  • 211
2 votes
0 answers
349 views

Google CloudSQL - cannot delete created role

I created user observer using only postgres user on my PostgreSQL 11 instance and I don't need it anymore. But after all I can't delete it - pgAdmin showing me that my user have connect privileges ...
Alexander Mikheev's user avatar
4 votes
2 answers
10k views

Efficient key value store in Postgres

I want to use Postgres(11 in CloudSQL) as an efficient key value store. I have about 200GB of dictionaries (average size is 10kB, the structure can be different and nested). I'm thinking about ...
Lukasz Madon's user avatar
1 vote
1 answer
255 views

migrate users from mariadb10.1 to mysql5.7

My company is moving our databases from self-hosted mariadb 10.1 to Google CloudSQL (which is essentially MySQL 5.7). Dumping / restoring our application schemas looks straightforward. However I'm ...
Mark S's user avatar
  • 122
2 votes
1 answer
4k views

PgAdmin 4.14 connection error in cloudSQL : could not send data to server, socket is not connected, could not send ssl negotiation packet

I am trying to connect cloud sql with pgadmin but it throws error when I try to execute query. "Could not send Data to server, socket is not connected, could not sent SSL negotiation pocket" ...
prabhu's user avatar
  • 23
0 votes
0 answers
165 views

ALTER TABLE causing transaction aborted

There are two transactions. One transaction I used BEGIN RO for read-only transaction. While the other one I used BEGIN to allow me adding a new column to my table called dummy_table. This is what I'...
user6308605's user avatar