Databases: Implementing MongoDB

Make sure you have read these articles prior to beginning:

• Databases: Overview and Introduction

• Databases: MongoDB Introduction

Now, let’s go through the steps to implement a MongoDB Database.

You can also refer to this documentation https://pymongo.readthedocs.io/en/stable/tutorial.html except for one step near the beginning:

• If you are using Repl or Heroku to connect to a MongoDB database in the cloud (which is typical), skip the step where it asks you to start up MongoDB with the mongod command

• The mongod command is only used when you are running your own local instance of MongoDB on your laptop, which is typically only useful for testing, not for implementing a real web application available on the web to others.

Step 1: Install pymongo Python Module

There is a Python module called Flask-PyMongo that we can install with pip install to make working with MongoDB easier.

To install it on Repl.it, use this in the Shell window:

python3 -m poetry add pymongo
python3 -m poetry lock

python3 -m poetry install

on Windows or Mac (if you're coding locally), use:

pip install pymongo

On Windows/Mac, if using pip, you’ll also need to make sure that you add these lines to the requirements.txt file for any webapp that you want to run on Heroku with MongoDB. You'll want to check that you have the latest versions---these version numbers are likely very out of date.

pymongo==3.11.0

Step 2: Create an account on MongoDB Atlas

Go to MongoDB Atlas and click the "Try Free" button on the top right. Put in all the necessary information and then create your account. You'll probably receive an email asking you to confirm your account.

On the next page, you'll be able to create your first cluster. Here, make sure to choose the "Shared Clusters", which is the free tier. Then move onto the next step.

On the next page, you should see "Create a Starter Cluster" at the top. If you do, you're in the right place. Scroll down to "Cloud Provider & Region", and select aws and Oregon (which is under North America. If you don't see Oregon, then just select the closest location to you). You can look at the screenshot below if you are confused. The rest of the options can be left the same, and go ahead and click the "Create Cluster" at the bottom of the page. It will take a few minutes for your cluster to be created.

Step 3: Adding databases to your new project

You've created a project in MongoDB's Atlas. Now, let's create a database and a collection.

Go to the main page of your projects, and select Clusters in the left navbar (we'll explain what clusters are in a bit). You should see something similar to the image below. Now click on the Collections button (look in the image, there's a red arrow pointing at the button.)

After clicking the Collections button, scroll down and search for a "Add my own data" button (should be in the middle of the screen once you scroll down a bit). Click that button, and then give a name for your database and collection.

• Database name can be anything that describes your program. In larger projects, it's sometimes better to split up your data into multiple databases (which is why MongoDB Atlas allows you to have multiple databases in one project), but one database should be good enough for your SPIS project.

• The collection name may very well be the same as the database name in many simple applications (e.g. cars, counties, etc.). Perhaps the best way to choose a collection name is think about what kind of “thing” in the real world is represented by each item in the collection. If it’s a collection representing movies, call it ‘movies’. Similar to Databases, it's sometimes better to split your data into multiple collections (i.e. Action, Romantic, Comedy movies). You're data might be simple enough to have a single collection though.

You've created a Database with a Collection! Congrats, now we'll go over how you can connect to the database. Before connecting, you need to add a database user. We'll go over this more in detail below.

Step 4: Connecting to your project

How can we connect to our database from our code?

The Python code for accessing an Atlas database has to have certain values in it. This is very similar to the variables used to connect to OAuth. Those values are:

• dbuser

• dbpassword

• hostname (e.g. ds135912.mlab.com) (Be sure to include the .mlab.com as well!)

• port number (35912)

• database name

These are all given to you in a short weird looking statement called the Mongo URI (Uniform Resource Identifier), which looks similar to a URL: mongodb://<dbuser>:<dbpassword>@YOURHOSTNAME.mlab.com:PORTNUMBER/DATABASE_NAME . We'll be using this in our code to access our database.

Getting the MongoDB URI

To get the MongoDB URI for your database, follow the steps below:

• Go to the main page of your projects, and select Clusters in the left navbar. You should see something similar to this:

• Now, click the connect button (look in the above picture for the red arrow). You should see something like this:

• Before being able to connect our application, we have to do a few things to setup connection security for MongoDB.

  • Click the green "Add Your Current IP Address" to add your IP Address.

  • Now, we will create the first database user, which will have admin permissions. Think of a simple username such as dbuser1. Note, this is not a human user, but rather a “machine user”, i.e. it is the user/password credentials that will be used by your Python application to connect to this database. Next, your password will be random characters such as weaf8jawel8f8waefjawe8fjlaw8fhalwifhaw3. Make sure to remember the password for the next steps -- consider writing it in a temporary local secure notepad that nobody but you will be able to see (once you create the database user, you're password will disappear).

    • Note: You'll either have to share this information with your partner, or you'll have to create another user for them (by clicking on Database Access in the left navbar, and then clicking the green "Add New Database User" button).

• Once you've done the above steps, click the green "Choose a connection method" button on the bottom of the popup. On the next page, there will be three large selections like this:

• Click the second large popup, which says "Connect your application". You should now see this:

• Note that the Driver (aka programming language) is Node.js, but we want it to be Python. Go ahead and change that to be Python, with the version of 3.6 or later. Now copy it to your clipboard.

Note: In this example, the MongoDB URI is mongodb+srv://test:<password>@cluster0.as3vh.mongodb.net/<dbname>?retryWrites=true&w=majority . Yours should have a similar format, but instead of test, you should have the username you filled out earlier listed there. There are a few things we have to type into the URI (i.e. <password>, <dbname>), but we'll do that in a bit.

If you are using Repl.it

Add a new "secret" in repl (similar to the instructions here for adding secrets for APP_SECRET_KEY). Call the secret MONGO_URI (not that's I as an India, not L as in Lima). And poste in the URL, such as
mongodb+srv://test:<password>@cluster0.as3vh.mongodb.net/?retryWrites=true&w=majority
as the value, but substituted in:

• the real password in place of <password> (take out the <> as well)

If you are using Heroku

Now that we have the MongoDB URI, let's connect our Heroku application to it:

• Go to your Heroku dashboard, and select your webapp.

• Go to Settings in the navbar, and scroll down to Config Vars. Click the "Reveal Config Vars" button. For the KEY, type MONGO_URI, and for the VALUE, type your MongoDB URI copied from earlier.

• Before saving, we have to change our MongoDB URI a bit.

  • Notice in the URI that you copied, there is a part that says <password> and <dbname> . Replace <password> with the password you created for your database user (there should not be any brackets <> afterwards). Replace <dbname> with the name of the database you'd like to access (checkout step 4 if you forgot where you put your database name) (there should not be any brackets <> afterwards).

  • In the example used earlier, if the password were to be pswrd12345 (a bad password to have) and the database name were to be database1, the MongoDB URI should look like this after replacing them: mongodb+srv://test:pswrd12345@cluster0.as3vh.mongodb.net/database1?retryWrites=true&w=majority

Now you have MongoDB Atlas set up with a database and collection, and it's connected to your Heroku account through the URI in the config variables.

Step 5: Implement MongoDB in your Python file

This implementation assumes that you have already implemented OAuth.

At the top of your file, we need an additional import statement for PyMongo in order to use MongoDB:

from pymongo import MongoClient

Next, include the following code in your Python file (probably near where you defined the OAuth variables):

client = MongoClient(os.environ['MONGO_URI')
db = client.NAME_OF_DATABASE

Change NAME_OF_DATABASE to whatever you named your database (not the collection, but the database) on the MongoDB web site.

Note the similar pattern we have implementing MongoDB as we do with OAuth. Heroku’s Config Vars are analogous to a system’s os environmental variables. When we host this on Heroku, your webapp will know to look in the Config Vars.

Below, we'll be explaining how to use Flask-PyMongo to do some basic database manipulations, but go to the documentation for the most information.

Adding Documents

The code for adding documents is:

db.NAME_OF_YOUR_COLLECTION.insert_one( INFO_YOU_ARE_INSERTING )

MongoDB has an unique way of adding information. It is very similar to the way dictionaries work in Python. Inside the INFO_YOU_ARE_INSERTING, there will be a set of keys and values that look like {key1: value1, key2: value2,... }. For example, if we were adding a document about a UCSD college student, the Python code might look something like this:

db.NAME_OF_YOUR_COLLECTION.insert_one({"Name": "FirstName LastName", "Major": "Computer Science", "College": "Warren", "Class": "2023"})

Finding Documents

The code for finding documents is:

db.NAME_OF_YOUR_COLLECTION.find( INFO_YOU_ARE_FINDING )

You might want to only find documents that meet a certain criteria. Using the example from above, perhaps we want to have a list of only the students that are in Sixth College. The code might look something like:

mongo.db.NAME_OF_YOUR_COLLECTION.find({"College": "Sixth"})

Both insert_one() and find() can take in numerous parameters that allow for specific use cases. Here is the link to the Flask-PyMongo documentation where you can learn about the methods available. I highly suggest researching on your own as these are just couple of the many things you can do. Finding additional resources might be beneficial for your final project.

An example web app running databases and OAuth will be put here soon for you to use as an example.

You now have all the tools necessary to build a fantastic web app. Be creative, look for inspiration, and be persistent with your work!