MongoDB is a document-oriented and NoSQL database solution that provides great scalability and flexibility along with a powerful querying system. With MongoDB and Python, you can develop many different types of database applications quickly. So if your Python application needs a database that’s just as flexible as the language itself, then MongoDB is for you. Show
In this tutorial, you’ll learn:
Throughout this tutorial, you’ll write a couple of examples that will demonstrate the flexibility and power of MongoDB and its great Python support. To download the source code for those examples, click the link below: Get the Source Code: Click here to get the source code you’ll use to learn about using MongoDB with Python in this tutorial. Using SQL vs NoSQL DatabasesFor decades, SQL databases were one of the only choices for developers looking to build large and scalable database systems. However, the increasing need for storing complex data structures led to the birth of NoSQL databases. This new kind of database system allows developers to store heterogeneous and structureless data efficiently. In general, NoSQL database systems store and retrieve data in a much different way from SQL (RDBMSs). When it comes to choosing from the currently available database technologies, you might need to decide between using SQL or NoSQL systems. Both of them have specific features that you should consider when choosing one or the other. Here are some of their more substantial differences: PropertySQL DatabasesNoSQL DatabasesData modelRelationalNonrelationalStructureTable-based, with columns and rowsDocument based, key-value pairs, graph, or wide-columnSchemaA predefined and strict schema in which every record (row) is of the same nature and possesses the same propertiesA dynamic schema or schemaless which means that records don’t need to be of the same natureQuery languageStructured Query Language (SQL)Varies from database to databaseScalabilityVerticalHorizontalACID transactionsSupportedSupported, depending on the specific NoSQL databaseAbility to add new propertiesNeed to alter the schema firstPossible without disturbing anything There are many other differences between the two types of databases, but those mentioned above are some of the more important ones to know about. When choosing a database, you should consider its strengths and weaknesses carefully. You also need to consider how the database fits into your specific scenario and your application’s requirements. Sometimes the right solution is to use a combination of SQL and NoSQL databases to handle different aspects of a broader system. Some common examples of SQL databases include:
NoSQL database examples include:
In recent years, SQL and NoSQL databases have even begun to merge. For example, database systems, such as PostgreSQL, MySQL, and Microsoft SQL Server now support storing and querying JSON data, much like NoSQL databases. With this, you can now achieve many of the same results with both technologies. But you still don’t get many of the NoSQL features, such as horizontal scaling and the user-friendly interface. With this brief background on SQL and NoSQL databases, you can focus on the main topic of this tutorial: the MongoDB database and how to use it in Python. Remove adsManaging NoSQL Databases With MongoDBMongoDB is a document-oriented database classified as NoSQL. It’s become popular throughout the industry in recent years and integrates extremely well with Python. Unlike traditional SQL RDBMSs, MongoDB uses collections of documents instead of tables of rows to organize and store data. MongoDB stores data in schemaless and flexible JSON-like documents. Here, schemaless means that you can have documents with a different set of fields in the same collection, without the need for satisfying a rigid table schema. You can change the structure of your documents and data over time, which results in a flexible system that allows you to quickly adapt to requirement changes without the need for a complex process of data migration. However, the trade-off in changing the structure of new documents is that exiting documents become inconsistent with the updated schema. So this is a topic that needs to be managed with care. Note: JSON stands for JavaScript Object Notation. It’s a file format with a human-readable structure consisting of key-value pairs that can be nested arbitrarily deep. MongoDB is written in C++ and actively developed by MongoDB Inc. It runs on all major platforms, such as macOS, Windows, Solaris, and most Linux distributions. In general, there are three main development goals behind the MongoDB database:
MongoDB is a distributed database, so high availability, horizontal scaling, and geographic distribution are built into the system. It stores data in flexible JSON-like documents. You can model these documents to map the objects in your applications, which makes it possible to work with your data effectively. MongoDB provides a powerful that supports ad hoc queries, indexing, aggregation, geospatial search, text search, and a lot more. This presents you with a powerful tool kit to access and work with your data. Finally, MongoDB is freely available and has great Python support. Reviewing MongoDB’s FeaturesSo far, you’ve learned what MongoDB is and what its main goals are. In this section, you’ll learn about some of MongoDB’s more important features. As for the database management side, MongoDB offers the following features:
As for the operations side, MongoDB provides a few tools and features that you won’t find in other database systems:
All these features are quite useful. For example, if you take advantage of the indexing feature, then much of your data will be kept in memory for quick retrieval. Even without indexing specific document keys, MongoDB caches quite a bit of data using the least recently used technique. Installing and Running MongoDBNow that you’re familiar with MongoDB, it’s time to get your hands dirty and start using it. But first, you need to install it on your machine. MongoDB’s official site provides two editions of the database server:
If you’re on Windows, then you can read through the installation tutorial for complete instructions. In general, you can go to the download page, select the Windows platform in the Available Downloads box, choose the 5 installer that fits your current system, and click Download.Run the installer and follow the on-screen instructions on the installation wizard. This page also provides information on how to . If you’re on macOS, then you can use Homebrew to install MongoDB on your system. See the installation tutorial to get the complete guide. Also, make sure to follow the instruction to . If you’re on Linux, then the installation process will depend on your specific distribution. For a detailed guide on how to install MongoDB on different Linux systems, go to the and select the tutorial that matches your current operating system. Make sure you run the MongoDB daemon, 6, at the end of the installation.Finally, you can also install MongoDB using Docker. This is handy if you don’t want to clutter your system with another installation. If you prefer this installation option, then you can read through the official tutorial and follow its directions. Note that previous knowledge of how to use Docker would be required in this case. With the MongoDB database installed and running on your system, you can start working with real databases using the 7 shell.Remove adsCreating MongoDB Databases With the > show dbs admin 0.000GB config 0.000GB local 0.000GB > 7 ShellIf you’ve followed the installation and running instructions, then you should already have an instance of MongoDB running on your system. Now you can start creating and testing your own databases. In this section, you’ll learn how to use the shell to create, read, update, and delete documents on a database. Running the > show dbs admin 0.000GB config 0.000GB local 0.000GB > 7 ShellThe 7 shell is an interactive JavaScript interface to MongoDB. You can use this tool to query and manipulate your data as well as to perform administrative operations. Since it’s a JavaScript interface, you won’t use the familiar SQL language to query the database. Instead, you’ll use JavaScript code.To launch the 7 shell, open your terminal or command line and run the following command:
This command takes you to the 7 shell. At this point, you’ll probably see a bunch of messages with information on the shell’s version and on the sever address and port. Finally, you’ll be presented with the shell prompt ( 4) to enter queries and commands.You can pass the database address as an argument to the 7 command. You can also use several options, such as specifying the host and port to access a remote database, and so on. For more details on how to use the 7 command, you can run 7.Establishing a ConnectionWhen you run the 7 command without arguments, it launches the shell and connects to the default local server provided by the 6 process at 0. This means you’re connected to the local host through port 1.By default, the 7 shell starts the session by establishing a connection to the 3 database. You can access the current database through the 4 object:
In this case, 4 holds a reference to 3, which is the default database. To switch databases, issue the command 7, providing a database name as an argument.For example, say you want to create a website to publish Python content, and you’re planning to use MongoDB to store your tutorials and articles. In that case, you can switch to the site’s database with the following command:
This command switches your connection to the 8 database. MongoDB doesn’t create the physical database file on the file system until you insert real data into the database. So in this case, 8 won’t show up in your current database list:
The 7 shell provides a lot of features and options. It allows you to query and manipulate your data and also to manage the database server itself.Instead of using a standardized query language such as SQL, the 7 shell uses the JavaScript programming language and a user-friendly API. This API allows you to play around with your data, which is the topic for the next section.Creating Collections and DocumentsA MongoDB is a physical container for of . Each database gets its own set of files on the file system. These files are managed by the MongoDB server, which can handle several databases. In MongoDB, a collection is a group of documents. Collections are somewhat analogous to tables in a traditional RDBMS, but without imposing a rigid schema. In theory, each document in a collection can have a completely different structure or set of fields. In practice, documents in a collection commonly share a similar structure to allow uniform retrieval, insertion, and update processes. You can enforce a uniform document structure by using document validation rules during updates and insertions. Allowing different document structures is a key feature of MongoDB collections. This feature provides flexibility and allows adding new fields to documents without the need for modifying a formal table schema. To create a collection using the 7 shell, you need to point 4 to your target database and then create the collections using the dot notation:
In this example, you use the dot notation to create 4 as a collection in 8, which is your current database. It’s important to note that MongoDB creates databases and collections lazily. In other words, they’re physically created only after you insert the first document.Once you have a database and a collection, you can start inserting documents. Documents are the unit of storage in MongoDB. In an RDBMS, this would be equivalent to a table row. However, MongoDB’s documents are way more versatile than rows because they can store complex information, such as , embedded documents, and even arrays of documents. MongoDB stores documents in a format called Binary JSON (), which is a binary representation of JSON. MongoDB’s documents are composed of field-and-value pairs and have the following structure:
The value of a field can be any BSON data type, including other documents, arrays, and arrays of documents. In practice, you’ll specify your documents using the JSON format. When you’re building a MongoDB database application, probably your most important decision is about the structure of documents. In other words, you’ll have to decide which fields and values your documents will have. In the case of the tutorials for your Python site, your documents might be structured like this:
A document is essentially a set of property names and their values. The values can be simple data types, such as strings and numbers, but they can also be arrays such as 6 in the above example.MongoDB’s document-oriented data model naturally represents complex data as a single object. This allows you to work with data objects holistically, without the need for looking at several places or tables. If you were using a traditional RDBMS to store your tutorials, then you would probably have a table to store your tutorials and another table to store your contributors. Then you’d have to set up a relationship between both tables so you could retrieve the data later on. Remove adsWorking With Collections and DocumentsSo far, you know the basics of how to run and use the 7 shell. You also know how to create your own documents using the JSON format. Now it’s time to learn how to insert documents into your MongoDB database.To insert a document into a database using the 7 shell, you first need to choose a collection and then call on the collection with your document as an argument:
With the first command, you switch to the database you want to use. The second command is a JavaScript method call that inserts a simple document into the selected collection, 4. Once you hit Enter, you get a message on your screen that informs you about the newly inserted document and its 1.Just like relational databases need a primary key to uniquely identify every row in a table, MongoDB documents need to have an 2 field that uniquely identifies the document. MongoDB allows you to enter a custom 2 as long as you guarantee its uniqueness. However, a widely accepted practice is to allow MongoDB to automatically insert an 2 for you.Similarly, you can add several documents in one go using :
Here, the call to 5 takes a list of tutorials and inserts them into the database. Again, the shell output shows information about the newly inserted documents and their automatically added 2 fields.The 7 shell also provides methods to perform , , and operations on the database. For example, you can use to retrieve the documents in a collection:
The first call to 9 retrieves all the documents in the 4 collection. On the other hand, the second call to 9 retrieves those tutorials that are authored by Joanna.With this background knowledge on how to use MongoDB through its 7 shell, you’re ready to start using MongoDB with Python. The next few sections will walk you through different options for using MongoDB databases in your Python applications.Using MongoDB With Python and PyMongoNow that you know what MongoDB is and how to create and manage databases using the 7 shell, you can start using MongoDB, but this time with Python. MongoDB provides an official Python driver called PyMongo.In this section, you’ll go through some examples that’ll help you get a feeling of how to use PyMongo to create your own database applications with MongoDB and Python. Each module within PyMongo is responsible for a set of operations on the database. You’ll have modules for at least the following tasks:
In general, PyMongo provides a rich set of tools that you can use to communicate with a MongoDB server. It provides functionality to query, retrieve results, write and delete data, and run database commands. Installing PyMongoTo start using PyMongo, you first need to install it in your Python environment. You can use a virtual environment, or you can use your system-wide Python installation, although the first option is preferred. PyMongo is available on PyPI, so the quickest way to install it is with 5. Fire up your terminal and run the following command: 0After a few downloads and other related steps, this command installs PyMongo on your Python environment. Note that if you don’t supply a specific version number, then 5 will install the latest available version.Note: For a complete guide on how to install PyMongo, check out the Installing/Upgrading page of its official documentation. Once you’re done with the installation, you can start a Python interactive session and run the following import: >>> 1If this runs without raising an exception in the Python shell, then your installation works just fine. If not, then carefully perform the steps again. Remove adsEstablishing a ConnectionTo establish a connection to a database, you need to create a instance. This class provides a client for a MongoDB instance or server. Each client object has a , which by default handles up to a hundred connections to the server. Get back to your Python interactive session and import 7 from 9. Then create a client object to communicate with your currently running MongoDB instance:>>> 2The code above establishes a connection to the default host ( 0) and port ( 1). 7 takes a set of arguments that allows you to specify custom host, port, and other connection parameters. For example, to provide a custom host and port, you can use the following code:>>> 3This is handy when you need to provide a 3 and 4 that differ from MongoDB’s default setup. You can also use the MongoDB URI format:>>> 4All these instances of 7 provide the same client setup to connect your current MongoDB instance. Which one you should use just depends on how explicit you want to be in your code.Once you’ve instantiated 7, you can use its instance to refer to that specific database connection, just like you did with the 7 shell’s 4 object in the above section.Working With Databases, Collections, and DocumentsOnce you have a connected instance of 7, you can access any database managed by the specified MongoDB server. To define which database you want to use, you can use the dot notation just like you did in the 7 shell:>>> 5In this case, 8 is the name of the database you’ll be working with. If the database doesn’t exist, then MongoDB creates it for you, but only when you perform the first operation on the database.You can also use if the name of the database isn’t a valid Python identifier: >>> 6This statement is handy when the name of your database isn’t a . For example, if your database is called 02, then you need to use dictionary-style access.Note: When you use the 7 shell, you have access to the database through the 4 global object. When you use PyMongo, you can assign the database to a variable called 4 to get similar behavior.Storing data in your database using PyMongo is similar to what you did with the 7 shell in the above sections. But first, you need to create your documents. In Python, you use dictionaries to create documents:>>> 7Once you’ve created the document as a dictionary, you need to specify which collection you want to use. To do that, you can use the dot notation on the database object: >>> 8In this case, 4 is an instance of and represents a physical collection of documents in your database. You can insert documents into 4 by calling on it with a document as an argument:>>> 9Here, 10 takes 12, inserts it into the 4 collection and returns an object. This object provides feedback on the inserted document. Note that since MongoDB generates the 15 dynamically, your output won’t match the 15 shown above.If you have many documents to add to the database, then you can use to insert them in one go: >>> 0This is faster and more straightforward than calling 10 multiple times. The call to 17 takes an iterable of documents and inserts them into the 4 collection in your 8 database. The method returns an instance of , which provides information on the inserted documents.To retrieve documents from a collection, you can use . Without arguments, 9 returns a object that yields the documents in the collection on demand:>>> 1Here, you run a loop on the object that 9 returns and print successive results, using to provide a user-friendly output format.You can also use 28 to retrieve a single document. In this case, you can use a dictionary that contains fields to match. For example, if you want to retrieve the first tutorial by Jon, then you can do something like this:>>> 2Note that the tutorial’s 15 is set under the 2 key, which is the unique document identifier that MongoDB automatically adds when you insert a document into your database.PyMongo also provides methods to , , and documents from a database. If you want to dive deeper into these features, then take a look at the documentation for 08.Remove adsClosing ConnectionsEstablishing a connection to a MongoDB database is typically an expensive operation. If you have an application that constantly retrieves and manipulates data in a MongoDB database, then you probably don’t want to be opening and closing the connection all the time since this might affect your application’s performance. In this kind of situation, you should keep your connection alive and only close it before exiting the application to clear all the acquired resources. You can close the connection by calling on the 7 instance:>>> 3Another situation is when you have an application that occasionally uses a MongoDB database. In this case, you might want to open the connection when needed and close it immediately after use for freeing the acquired resources. A consistent approach to this problem would be to use the 34 statement. Yes, 7 implements the :>>> 4If you use the 34 statement to handle your MongoDB client, then at the end of the 34 code block, the client’s 38 method gets called, which at the same time closes the connection by calling 32.Using MongoDB With Python and MongoEngineWhile PyMongo is a great and powerful Python driver for interfacing with MongoDB, it’s probably a bit too low-level for many of your projects. With PyMongo, you’ll have to write a lot of code to consistently insert, retrieve, update, and delete documents. One library that provides a higher abstraction on top of PyMongo is MongoEngine. MongoEngine is an object-document mapper (ODM), which is roughly equivalent to an SQL-based object-relational mapper (ORM). MongoEngine provides a class-based abstraction, so all the models you create are classes. Installing MongoEngineThere are a handful of Python libraries to help you work with MongoDB. MongoEngine, however, is a popular one that provides a nice set of features, flexibility, and community support. MongoEngine is available on PyPI. You can install it using the following 5 command: 5Once you’ve installed MongoEngine into your Python environment, you’re ready to start working with MongoDB databases using Python’s object-oriented features. The next step is to connect to your running MongoDB instance. Establishing a ConnectionTo establish a connection with your database, you need to use . This function takes several arguments. However, in this tutorial, you’ll use only three of them. Within your Python interactive session, type the following code: >>> 6Here, you first set the database name 4 to 43, which is the name of the database you want to work in. Then you provide a 3 and a 4 to connect to your current MongoDB instance. Since you’re using the default 3 and 4, you can omit these two parameters and just use 48.Working With Collections and DocumentsTo create documents with MongoEngine, you first need to define what data you want the documents to have. In other words, you need to define a . MongoEngine encourages you to define a document schema to help you reduce coding errors and to allow you to define utility or helper methods. Similar to ORMs, ODMs like MongoEngine provide a base or model class for you to define a document schema. In ORMs, that class is equivalent to a table, and its instances are equivalent to rows. In MongoEngine, the class is equivalent to a collection, and its instances are equivalent to documents. To create a model, you need to subclass and provide the required fields as . To continue with the blog example, here’s how you can create a model for your tutorials: >>> 7With this model, you tell MongoEngine that you expect a 50 document to have a 51, an 52, a list of 53, and a 54. The base class, 49, uses that information along with the field types to validate the input data for you.Note: One of the more difficult tasks with database models is data validation. How do you make sure that the input data conforms to your format requirements? That’s one of the reasons for you to have a coherent and uniform document schema. MongoDB is said to be a schemaless database, but that doesn’t mean it’s schema free. Having documents with a different schema within the same collection can lead to processing errors and inconsistent behavior. For example, if you try to save a 50 object without a 51, then your model throws an exception and lets you know. You can take this even further and add more restrictions, such as the length of the 51, and so on.There are a few general parameters that you can use to validate fields. Here are some of the more commonly used parameters:
Each specific field type also has its own set of parameters. You can check the for a complete guide to the available field types. To save a document to your database, you need to call on a document object. If the document already exists, then all the changes will be applied to the existing document. If the document doesn’t exist, then it’ll be created. Here’s an example of creating and saving a tutorial into your sample tutorials database: >>> 8By default, 63 inserts the new document into a collection named after the model class, 50, except using lowercase letters. In this case, the collection name is 4, which matches the collection you’ve been using to save your tutorials.PyMongo performs data validation when you call 63. This means that it checks the input data against the schema you declared in the 50 model class. If the input data violates the schema or any of its constraints, then you get an exception, and the data isn’t saved into the database.For example, here’s what happens if you try to save a tutorial without providing a 51:>>> 9In this example, first note that you can also build a 50 object by assigning values to its attributes. Second, since you don’t provide a 51 for the new tutorial, 63 raises a 73 telling you that the 51 field is required. Having automatic data validation is a great feature that will save you some headaches.Each 49 subclass has an 76 attribute that you can use to access the documents in the associated collection. For example, here’s how you can print the 51 of all your current tutorials:>>> 0The 78 loop iterates over all your tutorials and prints their 51 data to the screen. You can also use 76 to filter your documents. For example, say you want to retrieve the tutorials authored by Alex. In that case, you can do something like this:>>> 1MongoEngine is well suited to manage your MongoDB databases for just about any type of application. Its features make it ideal for creating efficient and scalable programs using a high-level approach. If you’re looking for more information about MongoEngine, be sure to check out its user guide. Remove adsConclusionIf you need a robust, scalable, and flexible database solution, then MongoDB might be a good option for you. MongoDB is a mature and popular NoSQL database with great Python support. With a good understanding of how to access MongoDB with Python, you’ll be ready to create database applications that scale well and provide excellent performance. With MongoDB, you also have the benefit of a human-readable and highly-flexible data model, so you can adapt to requirement changes quickly. In this tutorial, you learned:
The examples you coded in this tutorial are available for download. To get their source code, click the link below: Get the Source Code: Click here to get the source code you’ll use to learn about using MongoDB with Python in this tutorial. Mark as Completed 🐍 Python Tricks 💌 Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team. Send Me Python Tricks » About Leodanis Pozo Ramos Leodanis is an industrial engineer who loves Python and software development. He's a self-taught Python developer with 6+ years of experience. He's an avid technical writer with a growing number of articles published on Real Python and other sites. » More about LeodanisEach tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are: Aldren Bartosz Joanna Jacob Master Real-World Python Skills With Unlimited Access to Real Python Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas: Level Up Your Python Skills » Master Real-World Python Skills Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas: Level Up Your Python Skills » What Do You Think? Rate this article: Tweet Share Share EmailWhat’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know. Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. and get answers to common questions in our support portal. |