Questions tagged [python]

Python is a powerful and ubiquitous language. When using a Python variant or library (e.g. Pandas, SQLAlchemy, pg8000, psycopg2), please include it in the tags. Questions using this tag should restrict themselves to interactions directly with the database tier. General python scripting questions should be directed to [StackOverflow](https://stackoverflow.com/questions/tagged/python)

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

how to run db.adminCommand() using pymongo for mongodb 3.6

I want to run this command and get it's output using pymongo module. db.adminCommand({getParameter: 1, logLevel: 1}) or say I want to run this command db.adminCommand( { setParameter: 1, ...
4
votes
1answer
673 views

Flask mongodb assertion 13 not authorized for query on flask_session.sessions

Need help resetting mongodb back to its original state. We have a flask app running with mongodb with authentication turned off. Running on Ubuntu with std Flask. We tried ALL sorts of combinations ...
0
votes
0answers
10 views

Storage fluctuates when creating a table as select from another table via python

My table "test_table2" holds around 12mil records and has 6 columns with small records (short strings, dates, etc.). The size of the table is cca 1 GB. When I'm running this query: create table db....
0
votes
1answer
7 views

SQL optional where clauses without exponentially large number of only slightly different queries

I'm working with postgres in python and I want to write a clear query that optionally allows WHERE clause subsetting, but doesn't require me writing an exponential explosion of whether the subsetting ...
0
votes
1answer
280 views

Find and Insert Missing data in Mongodb Collection

I want to write python3 code to check and insert missing data. My MongoDB collection documents have one field named "height" which is BTC block number. I want to traverse in a range start to the ...
0
votes
0answers
10 views

Try to run Python programm from command line. ImportError: No module named mysql.connector

I'm new in programming. So I have a question about this error. I'm trying to run programm from command line in PyCharm, and get this Error. root@kali:~# python /root/PycharmProjects/lesson/ProL3/ner....
0
votes
1answer
19 views

Getting proper output of my SELECT clause. SQLite

I am coding a bus connection search engine. My goal is to see the time when the bus is departing, while the stop_from and stop_to names are provided. I have created the STOP table CREATE TABLE STOP (...
0
votes
0answers
18 views

PostgreSQL limiting number of rows for a certain condition

arena_stat_id | user_id | date_set | level | trophies ---------------+--------------------+----------------------------+-------+---------- 1 | ...
0
votes
2answers
726 views

Psycopg2 Errors on SQL statement when trying to copy data from CSV file into PostgreSQL database

I am not a developer or PostgreSQL DB admin, so this may be basic questions. Logistics: Windows 10 server / pgAdmin 4 / Postgres 10 / Python 2.7.13 I'm using a python script to ingest external data, ...
3
votes
1answer
922 views

select two tables and print the columns side by side with matching key in hive

I have the below tables: table_1 name_1 id_1 value_1 avi 1 21 adi 2 X table_2 name_2 id_2 value_2 avi 1 21 adi 2 22 metadata_info: can be read from ...
1
vote
1answer
28 views

Unable to convert a table to hypertable due to unique index error

I am creating a table using Flask SQL Alchemy as follows: class Price(db.Model): __tablename__ = "prices" id = db.Column(db.Integer, primary_key=True) country_code = db.Column(db.String(2)...
0
votes
1answer
473 views

get row/value by primary key number

I'm writing my own python wrapper on top of official mysqldb connector. I'm trying to design method that would dynamically extract row data knowing KEY ID. SELECT * FROM cameras; +----------+--------...
1
vote
1answer
22 views

SQLite where condition - compare two last rows

I have a SQLite DB table like this: [Stock_price_Table]: +-----+-------------+--------+------------+--------+--------------+----------------------+ |code | date | first | open | close ...
0
votes
1answer
1k views

Query using $gte and New Date() working in MongoDB shell but not through python pymongo

When I run the following in the mongodb shell, it returns the correct number of records: (searching for the matching "SourceName" within the last 8 days) mongos> db.fdl.count({ SourceName : "SRC1"...
0
votes
1answer
13 views

heroku postgres sql changes not reflected in db [closed]

I wrote this program import psycopg2 try: #connect to db cursor = connection.cursor() except Exception as e: raise e else: print("connection Success") cmd = "" while(True): ...
0
votes
0answers
4 views

Kafka producer for API results

I'm dealing with an application that has an API you can call that returns JSON results My question is how to produce these results into Kafka Would a JSON producer work fine and be best practice? ...
0
votes
1answer
28 views

Error using convert MySQL in Python

I'm practically newbie with MySQL. I'm trying to do a query to retrieve a preformatted date, obtained from a subtraction, using CONVERT. This is the query: sql = f"SELECT CONVERT(timestamp - {...
1
vote
1answer
1k views

Connecting client application to MariaDB Galera cluster

I currently have a REST API written in Flask that connects to a MariaDB server. I'm thinking about replacing the server with a Galera cluster to improve availability and ensure continuity in case one ...
1
vote
2answers
23 views

How do I return a json object of my database table [closed]

I'm creating a web client that uses data from my database to plot markers on a google map. So my collumns are id, name, address, lat, lon, description. My client gets the data from a route in my ...
1
vote
2answers
928 views

Calling MySQL stored procedure in python with field names

I am trying to call a MySQL stored procedure that returns a single row, I can get the values back back using cursor.callproc() and cursor.stored_results(), but I can't get name of the field back '...
1
vote
1answer
247 views

Alter postgres password from python?

When I run this query from psql shell everything works fine: ALTER ROLE postgres WITH ENCRYPTED PASSWORD 'somePassword'; but when I try exactly the same query from python (with psycopg2 package) ...
0
votes
1answer
229 views

CSV (UTF-8) to Postgres database (UTF-8), doesn't show äöå characters

I have csv-file that is UTF-8 encoded. I'm using python (psycopg2) to copy it into a Postgres database. My Postgres database is using UTF-8. However I have problems to show Ä, Ö and Å characters. If ...
0
votes
1answer
103 views

InnoDB Concurrent writes being ignored

Sorry if this isn't the right place, this is my first time posting anything here. Anyway, I know I'm not supposed to but I can think of no other way. I need to use a small MySQL table (70 rows x 6 ...
3
votes
2answers
3k views

Using AWS DynamoDB Geospatial index with Python

Amazon's AWS DynamoDB features Geospatial Indexing, which facilitates geo queries: Query Support: Box queries return items that fall within a pair of geo points that define a rectangle as ...
0
votes
1answer
73 views

MySQL 'generator' not closing connections

I have the following function which is leaving connections open but I can't figure out why. The Cursor class creates a python-generator type function, allowing me to iterate over millions of rows. ...
0
votes
0answers
24 views

MongoDB - Update all documents NOT returned in aggregate query

I'm using PyMongo to write a Mongo update script. I have an aggregate query which returns a sum of all searches performed by each user in the last 30 days: cursor = mongo_search_logs_collection....
0
votes
0answers
8 views

Pymongo: Query data from specific wiredtiger (.wt) files in mongodb

I am new in pymongo. I have some wiredtiger(.wt) available in mongodb as below. I wanted to know what data are being stored in each of the following (.wt) file. All these files are belongs to one ...
0
votes
0answers
7 views

New dataframe for different groups

I have a dataframe with a column 'Country'. Every country has multiple records. I want to write a function in python using pandas so that for every country it can return a seperate dataframe. Having ...
1
vote
1answer
19 views

Speed up AWS PostgreSQL insertion via Python script

I have 120,000 csv inside my AWS EC2 instance, each containing 120,000 rows. I can't insert each of them as is into my AWS RDS postgresql DB, that will be 120,000^2 = 14,400,000,000 records. Each csv ...
1
vote
1answer
144 views

No output when use pymongo watch()

I try to access real-time data changes with pymongo.mongo_client.MongoClient.watch(), code example (Python 3.5.3, pymongo==3.7.2): from pymongo import MongoClient DB_CON_STR = "mongodb://10.10.10....
1
vote
1answer
104 views

How to see unclosed PostgreSQL connection while using psycopg2? [closed]

I run python code: con = psycopg2.connect(some_dsn) cur = con.cursor() cur.execute(some_sql) res = cur.fetchone() #con.close() <-- connection is not closed Then I try: ps auxf | grep postgres ...
2
votes
1answer
317 views

Have several 128 elements lists, need to obtain distance between them

Is storing them in a CUBE type field and then getting euclidean distance, (using <->) the only way? I can not save them as a list of floats, since i get: "can't adapt type numpyndarray" Can ...
1
vote
0answers
18 views

How to migrate data from a large amount of csv files with time x space measurements to a database that needs to be available for analysis [closed]

I have made a simulation experiment to obtain time series measurements at different points along a power line. These measurements were done for different types of power lines in different test ...
1
vote
1answer
44 views

Storing many pandas DataFrames in SQLite with metadata

I need some help with designing a database. My aim is to persistently store a number of pandas DataFrames in a searchable way, and from what I've read SQLite is perfect for this task. Each DataFrame ...
0
votes
0answers
20 views

Populate mysql db to Shotgun server using Python

I've an internal mysql database. On each creation, modification or deletion of the content of this db, I need to run a python script to replicate those information into another software web server. ...
0
votes
2answers
26 views

inserting multiple values into SQL with %

This works: cursor = conn.execute("""Select FAMILY,F_NAME from birthday where B_MONTH = '%s' """ % (currentMonth)) I cannot seem to expand it to work using an AND clause to also include day, I get a ...
0
votes
0answers
231 views

Error: duplicate key value violates unique constraint, but the column does not have a unique constraint? postgreSQL

By the way, I'm trying to automate the below using python, but I've isolated the SQL code and tried it by itself on pgAdmin III, and it still doesn't work. So I have a table called tableName, and I ...
0
votes
1answer
271 views

Use of MySQL trigger to insert row's week number

I have a table of orders in a MariaDB database that currently has a datetime column. For the dashboard I am building for this database, I have a statistic I would like to display which is "Orders This ...
0
votes
1answer
342 views

Do I have to commit after an exception has been raised?

My database generates a lot of orphan pages, and I'm trying to figure out why. I often do this in my python code, using the fdb driver: cursor = connection.cursor() for item in list: try: ...
0
votes
3answers
20 views

Overwriting MySQL database to only store 1 month of data

We are logging data on hardware with only small memory (short on disk storage) only 4GB. We only require the data to be stored for 1 month and then be over written in a way that it overrides older ...
0
votes
1answer
32 views

Generated DDL doesn't match with CREATE TABLE statement

I'm using MySQL with InnoDB engine. I create a new table that contains a foreign key, eg: CREATE TABLE rooms ( id INTEGER NOT NULL AUTO_INCREMENT, my_id VARCHAR(15), house_id INTEGER, ...
-4
votes
1answer
80 views

how to get all the table existing in a database [closed]

import pandas as pd import sqlite3 conn = sqlite3.connect('Db-IMDB.db') imdb = pd.read_sql_query('''SELECT table_name FROM information_schema.tables''',conn) but it is giving error: DatabaseError: ...
0
votes
1answer
28 views

MongoDB | Problem inserting embedded documents into document

I am trying to insert data in a nested format. A snippet of my models (there are many, many more): class LifetimeMode(db.EmbeddedDocument): meta = {'collection': 'lifetime_mode'} career = ...
0
votes
2answers
299 views

Run SQL query with python in SQL Server raises error: SqlSatelliteCall error: Unsupported input data type in column

I am trying to run Python in SQL Server and want to define a variable which contains the result of a simple select query. exec sp_execute_external_script @language = N'python', @script = N' import ...
0
votes
0answers
160 views

SSHtunnelForwarder to Linux Box, then connect to Postgres RDS (AWS)

I'm trying to write a python script to update DDL in a Aurora RDS (postgres) on AWS. In Order to do this, I need to SSHTunnel to a linux box that sits at the edge of the VPC, then do another jump to ...
-2
votes
2answers
53 views

Python - Best database for exponentially growing database? [closed]

So I am making a Discord (social media) Bot in Python using discord.py-rewrite, and I need to save informations of lots of servers and users (such as how much money they have, what items thay have…). ...
0
votes
0answers
31 views

Neo4j: Connection closed after few minutes: defunct connection Address (Python)

I have a server (hosted in Azure) in which the connection is opened (python): self._driver: Driver = GraphDatabase.driver(credentials["uri"], auth=(credentials["user"], credentials["password"]), ...
0
votes
1answer
26 views

Inserting records into Mongodb using Pymongo is using up my RAM incrementally and is eventually exhausting all memory

I am writing Python code that parses through a huge XML document of size 17GB(Wikipedia collection) and inserts each page title along with the links in that page (storing as a list) present on the ...
1
vote
2answers
17k views

Unable to import mysql.connector in a Python venv session

I've used venv to create a virtual environment for Python 3.5.2. I have installed the mysql.connector. The connector works fine as long as I am not in an activated venv environment. from mysql....
0
votes
1answer
23 views

Is it necessary to connect to a postgresql database in order to create a new server?

I am writing a script in python 3.7, using psycopg2 to set up a new database from scratch. In all the tutorials I have seen, using psycopg2 starts by connecting to a database and getting a cursor ...