Requesting Crypto Prices from the Coinmarketcap REST API using Python

CoinMarketCap is one of the largest sources of cryptocurrency price information on the internet. The website offers multiple API endpoints that provide access to its exciting data, including current and historical price data for various cryptocurrencies such as Bitcoin, Ethereum, etc., and meta-information about coins and crypto exchanges. This tutorial will show how to use Python to request data from the Coinmarketcap API and use it for analytics. We will also store the data in a local SQLite database using the Pewee SQL framework for later use in analytics cases.

Coinmarketcap offers a REST API through which we can access Crypto Price data.

The rest of this article is a Python hands-on tutorial: First, we lay the groundwork for working with the Coinmarketcap API by signing up for a free API key. Subsequently, we store this in a YAML file and use it to query bitcoin quotes from the Coinmarketcap API endpoint. Our goal is to store the data with Peewee in an SQLite DB. We will first define a schema to store the data. Finally, we will learn how to query the obtained rates from our local database and use them later, for example, to train a price prediction model.

The Coinmarketcap API Endpoints

Coinmarketcap offers different API endpoints for various crypto-related data, including prices, exchanges, and other information. The price data ranges back to 2013. Below is a selection:

Endpoint When to Use?
/cryptocurrency/*To retrieve price and volume data (aggregated over all exchanges) or cryptocurrency-related price data.
/exchange/*To return exchange-specific data such as ordered exchange lists and market pair data.
/global-metrics/*To return, obtain aggregate statistics such as global market cap and BTC dominance.
/blockchain/*For obtaining blockchain analytics data, e.g., transaction volume, etc.
/key/*For managing and monitoring API key usage.

Authentication

All requests to the coinmarketcap API require authentication with an API key. Authentication can be achieved in two ways:

  • Send the API key in the custom header of the HTTP request with the attribute ‘X-CMC_PRO_API_KEY’ (Prefered).
  • Provide the key in the URL path.

Acquiring an API Key from the Developer Portal

Before requesting data from the coinmarketcap API, you need to acquire your API key by registering on the coinmarketcap API developer portal. You will choose from one of several pricing plans during the registration process. The Free basic plan provides access to most data and is sufficient for this tutorial. However, it has severe limitations, such as a limited number of requests you can make per day and month. After the registration, you can log in to your account and display the API key in the “Overview” section.

Each call to the Coinmarketcap API consumes API credits. The number of calls and data we can retrieve via API is limited to 100 credits for the free plan. You can see how many of your daily and monthly credits are still available on the overview page.

coinmarketcap api pricing plans, python rest api tutorial
Different pricing plans of the Coinmarketcap API
coinmarketcap api key
Overview page of the coinmarketcap API

Storing the API Key

It would be best to never store an API key directly in the code for security reasons. A better practice is to import and access the API key from a separate YAML file. Create the file with the name “api_config_coinmarketcap.yml” and insert your API key into this file as follow:

key: “your coinmarketcap api key”Python

Example: Obtaining Bitcoin Prices via the Coinmarketcap API

In the following, we write some Python code to send requests to the Coinmarketcap API at regular intervals. Our API requests obtain cryptocurrency prices for the number one in the market capitalization ranking. Each request to the Coinmarketcap API consumes credits. Therefore, our goal is to store the price data in a local SQLite database. We can then later retrieve it from there without consuming credits. To manage the SQLite DB, we use the Pewee framework.

The code is available on the GitHub repository.

Prerequisites

Before beginning the coding part, please make sure that you have set up your Python 3 environment and required packages. If you don’t have a Python environment available yet, you can follow the steps in this article to set up the Anaconda Python environment.

Also, make sure you install the required packages. In this tutorial, we will be working with the following standard packages: 

In addition, we will use Peewee, a lightweight Object-Relational-Mapper (ORM) framework that lets us interact with SQLite in an object-oriented and more convenient way.

You can install packages using console commands:

  • pip install <package name>
  • conda install <package name> (if you are using the anaconda packet manager)

Step #1 Create a Relational Model in SQLite

We begin by defining a relational data model in SQlite using Pewee. We will then use this model to persist the crypto price data from the API and make them available for later reuse. Our model will contain two tables:

  • Run: A table where we will store an ID and a timestamp make identifying past runs easier. Everything we start querying data from the API, we will add a new runid to this table.
  • The second table is called price and contains the price quotes for a predetermined interval. We will store the price quotes in this table and reference the runid via a foreign key.

Running the code below creates the empty tables.

from peewee import *
from datetime import date, timedelta, datetime
import numpy as np 
import pandas as pd
import seaborn as sns
import time
import json
import yaml
from requests import Request, Session
from requests.exceptions import ConnectionError, Timeout, TooManyRedirects
from pandas.plotting import register_matplotlib_converters

import matplotlib.dates as mdates
import matplotlib.pyplot as plt

# persist information
db = SqliteDatabase('assets.db')

class BaseModel(Model):
    class Meta:
        database = db

class Run(BaseModel): # This model uses the "assets.db" database.
    timestamp = DateTimeField(unique=True)
    symbol = CharField()
    id = AutoField()

class Price(BaseModel): # This model uses the "assets.db" database.
    datetime = DateTimeField(unique=True)
    volume_24 = FloatField()
    price = FloatField()
    runid = ForeignKeyField(Run, backref='prices')
         
# By default, Peewee includes an IF NOT EXISTS clause when creating tables. 
def create_tables():
    with db:
        db.create_tables([Price])
        db.create_tables([Run])

def drop_tables():
    with db:
        db.drop_tables([Price])
        db.drop_tables([Run])
        
create_tables()

#drop_tables()    

If you want to drop the tables again, you can call our custom drop_tables function.

Step #2 Streaming Price Quotes

Now that we have created the relational data model, we can define a request to the Coinmarketcap API and initiate the stream of price data. For this purpose, we will be calling the coinmarketcap REST endpoint “cryptocurrency/listings/latest.” This endpoint returns a JSON response that contains the latest price data for a specific number of cryptocurrencies based on their market capitalization rank. The response contains the following information for each cryptocurrency included:

  • Price value
  • The timestamp
  • The trading volume
  • The cost of the request in credits

Before querying the API, we need to define basic API parameters:

  • Conversion_Currency: The currency in which the API will return the prices.
  • Start: The position in the market cap ranking from which the API will include cryptocurrencies in the result.
  • Limit: The number of cryptocurrencies for which the API will return prices based on the start position in the current market cap ranking.

We will select USD as “conversion_currency,” set the limit, and start to “one.” As a result, the API will return only the price of Bitcoin. We use a query interval of 10 seconds. You can change the interval, but be aware that you will deplete your free credits quickly if you request data by the second. We will therefore limit the requests to 200. You can increase the timeframe to retrieve price quotes for more extended periods.

We will store and load our API key from a YAML file. Be sure to replace the api_key_path in the code below with the path where you placed the api_config_coinmarketcap.yml file. Once you have made this adjustment, you can start the API requests by running the code below.

symbol='BTCUSD'
query_interval = 10 # in seconds
query_number = 100 # the number of queries after which the API stops

# load the API key from our local file
with open(r'api_keys/api_config_coinmarketcap.yml') as file:
    apikey = yaml.load(file, Loader=yaml.FullLoader)['key']

# Define some essential API parameters
# Coinmarketcap API for latest market ticker quotes and averages for cryptocurrencies and exchanges.
# https://coinmarketcap.com/api/documentation/v1/#operation/getV1CryptocurrencyListingsLatest
url = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest'
parameters = {
  'start':'1',
  'limit':'1',
  'convert':'USD'
}
headers = {
  'Accepts': 'application/json',
  'X-CMC_PRO_API_KEY': apikey,
}

session = Session()
session.headers.update(headers)

# create a new run and save it to our local SQLite DB
run_timestamp = datetime.now()
run = Run(symbol=symbol, timestamp = run_timestamp)
run.save()
current_run_id = run.id 
print(f'run_id: {current_run_id} - timestamp: {run_timestamp} - interval: {query_interval} - number of queries: {query_number}')

# query the coinmarketcap API every x seconds
for s in range(0, query_number):
    try:
        response = session.get(url, params=parameters)
        data = json.loads(response.text)
        #print(data)
        
        # response - quote
        data_quote = data['data'][0]['quote']['USD']
        price = np.round(data_quote['price'], 1) # the price
        volume_24 = np.round(data_quote['volume_24h'], 1) # the volume in the last 24 hours
        
        # response - status
        data_status = data['status']
        api_timestamp = data_status['timestamp'] # the timestamp of the pricepoint
        api_credits = data_status['credit_count'] # the number of credits spent on the last request
        
        # create a new pricepoint and save it to the SQLite db
        new_pricepoint = Price(datetime=api_timestamp, price=price, volume_24=volume_24, runid=current_run_id)
        id = new_pricepoint.save()

        # display what we have just saved
        print(f'request number: {s} - added {price} at {api_timestamp} - 24 hour volume: {volume_24} - credits used: {api_credits}')      

    except (ConnectionError, Timeout, TooManyRedirects) as e:
        print(e)
    time.sleep(query_interval)
print('finished')
status response returned by the coinmarketcap REST API

Step #3 Querying the Data from Our SQL Table

We now have the price data persisted in our local SQLite DB. From there, we can query the data by using the peewee “select” SQL command. For example, we will query the run table and retrieve the number of requests made to the coinmarketcap API.

query = Run.select().where(Run.id==current_run_id)
run_overview = pd.DataFrame(list(query.dicts()))
run_overview

Next, select the price quotes for this specific run-id and display them in a line plot.

query = Price.select().where(Price.runid==current_run_id)
df_prices = pd.DataFrame(list(query.dicts()))
print(df_prices)

register_matplotlib_converters()
fig, ax = plt.subplots(figsize=(10, 8))
plt.title('BTC prices from the coinmarketcap API', fontsize=14)
datetimes = pd.to_datetime(df_prices['datetime'])

sns.lineplot(data=df_prices, x=datetimes, y="price")
#ax.set_xlim([df_prices['datetime'].min(),df_prices['datetime'].max()])
#ax.xaxis.set_major_locator(mdates.MinuteLocator())
plt.xticks(rotation=75)
price quotes obtained from the coinmarketcap api

Summary

In this tutorial, you have learned how to request crypto price data from the Coinmarketcap REST API. We wrote a short Python script that queries one of several REST endpoints (cryptocurrency/listings/latest) in regular intervals. To ensure that we can later reuse the requested data, we stored the response from the API in an SQLite database using the Pewee framework.

If the tutorial helped you or you have any comments, please let me know in the comments.

If you are not sure yet, if the coinmarketcap API is for you, consider these other ways of obtaining cryptocurrency price data:

And if you are interested in stock-market prediction, check out the following Python tutorials:

Author

  • Hi, I am Florian, a Zurich-based consultant for AI and Data. Since the completion of my Ph.D. in 2017, I have been working on the design and implementation of ML use cases in the Swiss financial sector. I started this blog in 2020 with the goal in mind to share my experiences and create a place where you can find key concepts of machine learning and materials that will allow you to kick-start your own Python projects.

Leave a Reply