Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Comments on Can pandas be used as a database backend for persistent storage?

Parent

Can pandas be used as a database backend for persistent storage?

+2
−1

Question

What is the current state of the art database app? How does it compare to SQL? Can pandas be used in place of either?

If not, is there something that bridges the gap between SQL and pandas or the current state of the art?

Context

I have become proficient in pandas and have come to like the syntax. I have experience with SQL, but would have to study my notes to get back up to speed, and find the syntax rather tedious.

History
Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+4
−0

You can use Pandas to_sql method to upload records to a relational database.

However, be aware this functionality is aimed at using Pandas for its original purpose (data analytics and transformations), so that Pandas can make complex processing and send the results to a SQL table, but it is not advisable to write a traditional "database app" using Pandas as the database connector, since it will be slow performing and less scalable (Pandas is not meant for high throughput applications).

You can use an ORM like Django ORM or SQLAlchemy if writing bare SQL is annoying for you.

History
Why does this post require moderator attention?
You might want to add some details to your flag.

1 comment thread

ORM is not noob-friendly (2 comments)
ORM is not noob-friendly
matthewsnyder‭ wrote 10 months ago · edited 10 months ago

Worth saying that ORMs are very complex, advanced things. They require advanced knowledge of both SQL and Python. If either is lacking, things will be very confusing.

Moreover, unless you are doing some very complicated stuff (eg. large codebase maintained over years), the ORM often creates more busywork than it takes away.

I really would not recommend an ORM as a way of escaping the need to learn SQL.

__blackjack__‭ wrote 10 months ago

I second the need to know SQL but ORMs also simplify stuff. It doesn't have to get very complicated to profit from an automatic mapping between objects and records in a database. It takes away the boilerplate code to do this manually or to write an abstraction which ends up being somewhat an ORM itself. Reinventing the wheel.

SQLAlchemy doesn't enforce the ORM though. There is core layer which allows to build SQL queries programmatically without the danger to introduce errors or even security risks, that often comes with handling all with just string operations that contain SQL fragments. Another benefit is the abstraction over differences in SQL syntax of RDBMS. Like which placeholders are used, which characters to quote names, if there is a native boolean type or not, and so on.

Pandas implies SQLAlchemy. The only guaranteed exception that can be used without it is SQLite3.