Questions tagged [python]

Python is a powerful and ubiquitous language. When using a Python variant or library (e.g. Pandas, SQLAlchemy, pg8000, psycopg2), please include it in the tags. Questions using this tag should restrict themselves to interactions directly with the database tier. General python scripting questions should be directed to [StackOverflow](https://stackoverflow.com/questions/tagged/python)

Filter by
Sorted by
Tagged with
1
vote
1answer
123 views

No output when use pymongo watch()

I try to access real-time data changes with pymongo.mongo_client.MongoClient.watch(), code example (Python 3.5.3, pymongo==3.7.2): from pymongo import MongoClient DB_CON_STR = "mongodb://10.10.10....
1
vote
1answer
29 views

How to see unclosed PostgreSQL connection while using psycopg2? [closed]

I run python code: con = psycopg2.connect(some_dsn) cur = con.cursor() cur.execute(some_sql) res = cur.fetchone() #con.close() <-- connection is not closed Then I try: ps auxf | grep postgres ...
2
votes
1answer
299 views

Have several 128 elements lists, need to obtain distance between them

Is storing them in a CUBE type field and then getting euclidean distance, (using <->) the only way? I can not save them as a list of floats, since i get: "can't adapt type numpyndarray" Can ...
0
votes
1answer
97 views

CSV (UTF-8) to Postgres database (UTF-8), doesn't show äöå characters

I have csv-file that is UTF-8 encoded. I'm using python (psycopg2) to copy it into a Postgres database. My Postgres database is using UTF-8. However I have problems to show Ä, Ö and Å characters. If ...
1
vote
0answers
17 views

How to migrate data from a large amount of csv files with time x space measurements to a database that needs to be available for analysis [closed]

I have made a simulation experiment to obtain time series measurements at different points along a power line. These measurements were done for different types of power lines in different test ...
1
vote
1answer
28 views

Storing many pandas DataFrames in SQLite with metadata

I need some help with designing a database. My aim is to persistently store a number of pandas DataFrames in a searchable way, and from what I've read SQLite is perfect for this task. Each DataFrame ...
0
votes
0answers
16 views

Populate mysql db to Shotgun server using Python

I've an internal mysql database. On each creation, modification or deletion of the content of this db, I need to run a python script to replicate those information into another software web server. ...
0
votes
0answers
11 views

How to perform Partition by in SQLAlchemy [migrated]

I am trying to use SQL alchemy & wanted to see how to create the partition by on a column, but i couldn't find anything for sqlalchemy in respect a declarative extention. For example, if I have ...
0
votes
2answers
25 views

inserting multiple values into SQL with %

This works: cursor = conn.execute("""Select FAMILY,F_NAME from birthday where B_MONTH = '%s' """ % (currentMonth)) I cannot seem to expand it to work using an AND clause to also include day, I get a ...
0
votes
0answers
53 views

Error: duplicate key value violates unique constraint, but the column does not have a unique constraint? postgreSQL

By the way, I'm trying to automate the below using python, but I've isolated the SQL code and tried it by itself on pgAdmin III, and it still doesn't work. So I have a table called tableName, and I ...
0
votes
1answer
265 views

Use of MySQL trigger to insert row's week number

I have a table of orders in a MariaDB database that currently has a datetime column. For the dashboard I am building for this database, I have a statistic I would like to display which is "Orders This ...
0
votes
1answer
306 views

Do I have to commit after an exception has been raised?

My database generates a lot of orphan pages, and I'm trying to figure out why. I often do this in my python code, using the fdb driver: cursor = connection.cursor() for item in list: try: ...
0
votes
3answers
17 views

Overwriting MySQL database to only store 1 month of data

We are logging data on hardware with only small memory (short on disk storage) only 4GB. We only require the data to be stored for 1 month and then be over written in a way that it overrides older ...
4
votes
1answer
630 views

Flask mongodb assertion 13 not authorized for query on flask_session.sessions

Need help resetting mongodb back to its original state. We have a flask app running with mongodb with authentication turned off. Running on Ubuntu with std Flask. We tried ALL sorts of combinations ...
0
votes
1answer
247 views

Find and Insert Missing data in Mongodb Collection

I want to write python3 code to check and insert missing data. My MongoDB collection documents have one field named "height" which is BTC block number. I want to traverse in a range start to the ...
0
votes
1answer
25 views

Generated DDL doesn't match with CREATE TABLE statement

I'm using MySQL with InnoDB engine. I create a new table that contains a foreign key, eg: CREATE TABLE rooms ( id INTEGER NOT NULL AUTO_INCREMENT, my_id VARCHAR(15), house_id INTEGER, ...
0
votes
1answer
594 views

Psycopg2 Errors on SQL statement when trying to copy data from CSV file into PostgreSQL database

I am not a developer or PostgreSQL DB admin, so this may be basic questions. Logistics: Windows 10 server / pgAdmin 4 / Postgres 10 / Python 2.7.13 I'm using a python script to ingest external data, ...
-4
votes
1answer
74 views

how to get all the table existing in a database [closed]

import pandas as pd import sqlite3 conn = sqlite3.connect('Db-IMDB.db') imdb = pd.read_sql_query('''SELECT table_name FROM information_schema.tables''',conn) but it is giving error: DatabaseError: ...
3
votes
1answer
902 views

select two tables and print the columns side by side with matching key in hive

I have the below tables: table_1 name_1 id_1 value_1 avi 1 21 adi 2 X table_2 name_2 id_2 value_2 avi 1 21 adi 2 22 metadata_info: can be read from ...
0
votes
1answer
1k views

Query using $gte and New Date() working in MongoDB shell but not through python pymongo

When I run the following in the mongodb shell, it returns the correct number of records: (searching for the matching "SourceName" within the last 8 days) mongos> db.fdl.count({ SourceName : "SRC1"...
0
votes
1answer
397 views

get row/value by primary key number

I'm writing my own python wrapper on top of official mysqldb connector. I'm trying to design method that would dynamically extract row data knowing KEY ID. SELECT * FROM cameras; +----------+--------...
1
vote
1answer
1k views

Connecting client application to MariaDB Galera cluster

I currently have a REST API written in Flask that connects to a MariaDB server. I'm thinking about replacing the server with a Galera cluster to improve availability and ensure continuity in case one ...
0
votes
1answer
25 views

MongoDB | Problem inserting embedded documents into document

I am trying to insert data in a nested format. A snippet of my models (there are many, many more): class LifetimeMode(db.EmbeddedDocument): meta = {'collection': 'lifetime_mode'} career = ...
0
votes
2answers
193 views

Run SQL query with python in SQL Server raises error: SqlSatelliteCall error: Unsupported input data type in column

I am trying to run Python in SQL Server and want to define a variable which contains the result of a simple select query. exec sp_execute_external_script @language = N'python', @script = N' import ...
0
votes
1answer
99 views

InnoDB Concurrent writes being ignored

Sorry if this isn't the right place, this is my first time posting anything here. Anyway, I know I'm not supposed to but I can think of no other way. I need to use a small MySQL table (70 rows x 6 ...
1
vote
1answer
230 views

Alter postgres password from python?

When I run this query from psql shell everything works fine: ALTER ROLE postgres WITH ENCRYPTED PASSWORD 'somePassword'; but when I try exactly the same query from python (with psycopg2 package) ...
0
votes
0answers
109 views

SSHtunnelForwarder to Linux Box, then connect to Postgres RDS (AWS)

I'm trying to write a python script to update DDL in a Aurora RDS (postgres) on AWS. In Order to do this, I need to SSHTunnel to a linux box that sits at the edge of the VPC, then do another jump to ...
-2
votes
2answers
49 views

Python - Best database for exponentially growing database? [closed]

So I am making a Discord (social media) Bot in Python using discord.py-rewrite, and I need to save informations of lots of servers and users (such as how much money they have, what items thay have…). ...
0
votes
0answers
17 views

Neo4j: Connection closed after few minutes: defunct connection Address (Python)

I have a server (hosted in Azure) in which the connection is opened (python): self._driver: Driver = GraphDatabase.driver(credentials["uri"], auth=(credentials["user"], credentials["password"]), ...
0
votes
1answer
25 views

Inserting records into Mongodb using Pymongo is using up my RAM incrementally and is eventually exhausting all memory

I am writing Python code that parses through a huge XML document of size 17GB(Wikipedia collection) and inserts each page title along with the links in that page (storing as a list) present on the ...
1
vote
2answers
16k views

Unable to import mysql.connector in a Python venv session

I've used venv to create a virtual environment for Python 3.5.2. I have installed the mysql.connector. The connector works fine as long as I am not in an activated venv environment. from mysql....
0
votes
1answer
23 views

Is it necessary to connect to a postgresql database in order to create a new server?

I am writing a script in python 3.7, using psycopg2 to set up a new database from scratch. In all the tutorials I have seen, using psycopg2 starts by connecting to a database and getting a cursor ...
1
vote
2answers
427 views

psycopg2 connection with multihosts [closed]

I have 3 db servers on docker containers, and i used to connect like this from my python app: psycopg2.connect(host="db-0", user="user", password="password", database="data") This works,but if i add ...
0
votes
1answer
186 views

write csv file only after SQL table update, insert, delete

Objective: Write csv file of updated data from sql table only after SQL table update, insert, delete. I've read from the community that I should not do triggers to execute python. I should not run ...
1
vote
0answers
63 views

Designing a table with precalculated team values for fast read queries

My version of PostgreSQL is 10.7. As it's something I can't calculate on the fly due to the amount of times it might be used, I decided to store all the 11 million combinations and their result on a ...
0
votes
0answers
18 views

How to increase idle time of “LaunchPad” for SQL Server external scripts?

sp_execute_external_script can take a while to initialize when not run for a short time. How do I set the idle time on the controller Launchpad process to keep it from shutting down too soon? Note I ...
11
votes
1answer
2k views

Why is PL/Python untrusted?

According to the docs: PL/Python is only available as an "untrusted" language, meaning it does not offer any way of restricting what users can do in it and is therefore named plpythonu. A trusted ...
0
votes
1answer
748 views

How to Bulk Insert sqlalchemy subclasses

I am trying to bulk insert SQL-Alchemy Subclasses into the parent table and their respective tables ie fruits tables -> Apple Table and so I insert a table of APPLE and it will insert both the row ...
2
votes
3answers
22k views

How to uninstall pgAdmin4 on Ubuntu 16.04?

I successfully installed pgAdmin4 with pip install pgadmin4-2.0-py2.py3-none-any.whl But afterward, pip list doesn't show anything related to pgAdmin, so I don't know what argument to pass to pip ...
1
vote
0answers
165 views

operand should contain 1 column Use Variables in SQL statement in Python

Trying to insert four columns and multiple rows with variables. The table in mysql has 8 columns in total. Four columns to insert is as following: report_type= tuple(("ada","asda","frfe","adaaa","...
1
vote
1answer
1k views

Python version issue on Cassandra 3.11.2

Searching around I found a lot of thread about this problem (coming from 2012) but I can't find an official solution: running cqlsh.bat File "C:\apache-cassandra-3.11.2\bin\\cqlsh.py", line 146 ...
0
votes
0answers
16 views

Reconstruct folder hierarchy in JSON format from MySQL using Python

I have a table with a structure like this: | itemID | parentFolderID | folderName 1 0 folderA 2 1 folderB 3 1 ...
0
votes
1answer
492 views

Python code in T-SQL environment gives “access a socket in a way forbidden by its access permissions” Error

Help on the following please. I have this code Python Code import requests r = requests.get("https://www.google.com") r.status_code print(r.status_code) print("Python Rawks!!!!!!") which ...
0
votes
0answers
88 views

which python version is required to install plpython3u of which postgresql version?

I want to know if there is a reference table or trick to find out which python version is required by plpython3u of which version of postgresql . example python 3.4 is required for plpython3u of ...
1
vote
0answers
16 views

Custom Export from power BI service

I'm trying to make a custom button that exports filtered tables to a csv and downloads them to my computer. I am aware that you can do this with the '...' button but this functionality is for other ...
0
votes
0answers
22 views

Changing Cube Max Dimension

thanks in advance for any help. Python/postgres noob here. I need to change the default dimension for cubes in postgres. I tried to download source code, fixing the cubedata.h file to change max ...
0
votes
1answer
86 views

MySQL: columns with mixed character encodings, and finding columns which have multibyte data

I have some rather large CSV files that I am loading into my MySQL 5.7 database. The files are several gigabytes in size, several million lines long, and have large column widths that must be used in ...
7
votes
3answers
38k views

How to work with multiple Postgres databases in psycopg2

I have two different Postgres databases on two different servers (one is actually local). Similar to this question, I would like to work with both databases at the same time. However, I can't figure ...
0
votes
0answers
426 views

psycopg2.pool.PoolError: trying to put unkeyed connection

I'm trying to get connection pooling to work in my Python Flask web application. The code I am using is as follows: from contextlib import contextmanager import logging from psycopg2 import pool, ...
0
votes
2answers
569 views

0kb backup using mysqldump

Same question has been asked here but its not helping me out. In my python utility, I'm trying to take MySQL database backup using mysqldump. It runs successfully but I'm getting 0kb file in my ...