API documentation

Walkthroughs for the tasks scripts usually need, each with curl, Python and R versions that run against this site.

Last reviewed against the server code. Every endpoint is listed in the API reference.

Before you start

The samples in these guides assume three things:

  • The tools are installed: curl and jq for the shell, pip install requests for Python, install.packages("httr2") for R.
  • Samples that need a token read it from BLUEPRINT_TOKEN. Set it as in the quickstart, or let your script sign in by itself.
  • Samples that sign in read BLUEPRINT_USER and BLUEPRINT_PASSWORD. Set them without leaving the password in your shell history or a notebook:
printf 'Username: '; read -r BLUEPRINT_USER
printf 'Password: '; read -rs BLUEPRINT_PASSWORD; echo
export BLUEPRINT_USER BLUEPRINT_PASSWORD

Input is hidden where it says so; press Enter to finish each line.

Sign in from a script

A token copied from your Profile page lasts 15 minutes by default. For anything longer, let the script sign in with your username and password, keep the refresh cookie, and refresh when a call answers 401.

Create the account on the website first; registration needs a code sent to your email address or phone. The sample reads BLUEPRINT_USER and BLUEPRINT_PASSWORD, set as in Before you start.

# Needs jq. The cookie jar holds the refresh token: it is a credential,
# so it lives in a private temporary file (mktemp creates it mode 0600).
BASE="https://blueprint.love"
COOKIES=$(mktemp)

login() {
  body=$(jq -n --arg u "$BLUEPRINT_USER" --arg p "$BLUEPRINT_PASSWORD" \
      '{username: $u, password: $p}' |
    curl -s -c "$COOKIES" -H "Content-Type: application/json" -d @- \
      "$BASE/api/auth/login")
  if echo "$body" | jq -e '.data.requires_2fa' >/dev/null; then
    echo "Two-factor account: finish with POST /api/auth/2fa/verify" >&2
    return 1
  else
    # Prints nothing, and says why, unless the sign-in worked.
    echo "$body" | jq -r '.data.access_token // ("Sign-in failed: " + .message + "\n" | halt_error)'
  fi
}

refresh() {
  curl -s -b "$COOKIES" -c "$COOKIES" -X POST "$BASE/api/auth/refresh" |
    jq -r '.data.access_token // ("Refresh failed: " + .message + "\n" | halt_error)'
}

BLUEPRINT_TOKEN=$(login) &&
  curl -s -H "Authorization: Bearer $BLUEPRINT_TOKEN" "$BASE/api/auth/me" |
  jq -r '.data.username'

# Later, when a call answers 401:
BLUEPRINT_TOKEN=$(refresh)
  • Sign-in allows 3 per second per IP address, and 10 failed attempts for one username or email address, each within 30 minutes of the one before, lock sign-in with it for 15 minutes. Sign in once and refresh; do not sign in before every call. A 401 from sign-in means wrong credentials: refreshing will not help.
  • Each refresh replaces the cookie. If two refreshes race, one answers 409: retry it.
  • With two-factor authentication on, the Python sample asks for your code; the curl and R samples stop and say so.
  • Treat the password and tokens as secrets. If a token leaks, sign out on the website: that ends every session and revokes the tokens issued before it, including the script's. If your password leaks, change it in Settings, which also ends every session.

Page through projects and samples

GET /api/projects is public. Ask for pages with page and pageSize, and stop at totalPages. Filter with program, primary_site or disease_type; the accepted values come from GET /api/projects/filters.

A project's samples, with clinical fields such as vital status and follow-up days, need a token and come at most 500 to a page.

# Needs jq.
BASE="https://blueprint.love"

page=1
while body=$(curl -fsS "$BASE/api/projects?program=TCGA&page=$page&pageSize=20"); do
  echo "$body" | jq -r '.data.items[] | [.id, .description, .case_count] | @tsv'
  [ "$page" -ge "$(echo "$body" | jq '.data.totalPages')" ] && break
  page=$((page + 1))
done  # -f stops the loop on an HTTP error instead of paging on forever

# Samples need a token, and a page holds at most 500.
curl -s -H "Authorization: Bearer $BLUEPRINT_TOKEN" \
  "$BASE/api/projects/TCGA-BRCA/samples?page=1&pageSize=100" |
  jq -r '.data.items[] | [.barcode, .sample_type, .vital_status] | @tsv'

A project's case_count counts samples, and one patient can give more than one, so it is not the number of patients.

Render a figure

Figures render as jobs. This one draws TP53 expression across TCGA-BRCA tissue types as a box plot, waits for it, and saves the PNG.

# Needs jq.
BASE="https://blueprint.love"
AUTH="Authorization: Bearer $BLUEPRINT_TOKEN"

RESP=$(curl -s -H "$AUTH" -H "Content-Type: application/json" -d '{
  "plotType": "boxplot", "projectId": "TCGA-BRCA", "geneId": "TP53",
  "x": "tissue_type", "y": "vstd",
  "colorScheme": "tab10", "width": 800, "height": 600, "format": "png"
}' "$BASE/api/plot/jobs")
JOB=$(echo "$RESP" | jq -r '.data.id // empty')
[ -z "$JOB" ] && echo "$RESP" | jq -r '.message'  # refused: says why

STATUS=
while [ -n "$JOB" ]; do
  STATUS=$(curl -s -H "$AUTH" "$BASE/api/plot/jobs/$JOB" | jq -r '.data.status')
  case "$STATUS" in queued|processing) sleep 2 ;; *) break ;; esac
done

if [ "$STATUS" = completed ]; then
  # -f: a 404 here means the figure is still being stored; run this line again.
  curl -fsS -H "$AUTH" -o tp53-brca.png "$BASE/api/plot/jobs/$JOB/result"
elif [ -n "$JOB" ]; then
  curl -s -H "$AUTH" "$BASE/api/plot/jobs/$JOB" | jq -r '.data.error_message'
fi
  • Poll no more often than every 2 seconds. A failed job's error_message says why.
  • Submissions allow 30 per minute, and you can hold 20 unfinished plot jobs by default.
  • format also takes svg and pdf for figures you will edit or print.

Run an analysis and export its results

This runs a PCA on TCGA-BRCA's variance-stabilised expression, prints the explained variance, and saves the sample scores as CSV.

# Needs jq.
BASE="https://blueprint.love"
AUTH="Authorization: Bearer $BLUEPRINT_TOKEN"

RESP=$(curl -s -H "$AUTH" -H "Content-Type: application/json" -d '{
  "analysisType": "pca", "projectId": "TCGA-BRCA", "expressionType": "vstd",
  "pca": {"nComponents": 5, "nTopGenes": 2000}
}' "$BASE/api/analysis/jobs")
JOB=$(echo "$RESP" | jq -r '.data.id // empty')
[ -z "$JOB" ] && echo "$RESP" | jq -r '.message'  # refused: says why

STATUS=
while [ -n "$JOB" ]; do
  STATUS=$(curl -s -H "$AUTH" "$BASE/api/analysis/jobs/$JOB" | jq -r '.data.status')
  case "$STATUS" in queued|processing) sleep 5 ;; *) break ;; esac
done

if [ "$STATUS" = completed ]; then
  curl -s -H "$AUTH" "$BASE/api/analysis/jobs/$JOB?include=result" |
    jq '.data.result.explainedVarianceRatio'
  curl -s -H "$AUTH" -o pca-scores.csv \
    "$BASE/api/analysis/jobs/$JOB/export/scores?format=csv"
elif [ -n "$JOB" ]; then
  curl -s -H "$AUTH" "$BASE/api/analysis/jobs/$JOB" | jq -r '.data.error_message'
fi
  • Differential expression (deseq2, limma, limma_voom, edger) runs the same way; download its table from /api/analysis/jobs/JOB_ID/results?format=tsv.
  • Sending the same request again, while your earlier job is queued, running or completed, returns that job with data.reused set to true. After a failed or cancelled job it runs again.
  • Submissions allow 10 per minute.

Analyse your own data

Upload a table and use the fileId it returns wherever a request takes projectId. Send a valid token with the upload. A missing, expired or revoked token is not refused: the file is stored anonymously and your jobs cannot use it, so refresh first if the token may have lapsed.

# Needs jq.
curl -s -H "Authorization: Bearer $BLUEPRINT_TOKEN" \
  -F "file=@expression.csv" "https://blueprint.love/api/upload" | jq -r '.data.fileId'
  • Files are limited to 50 MiB by default. Uploading the same file again returns the same fileId.
  • Use delimited text such as CSV or TSV, laid out the way the figure expects. Each plot form on the Analysis pages shows that layout, and the catalogue plots publish example files under /plot-samples/.

Download published TCGA files

The catalogue lists each cohort's count, TPM and FPKM matrices and its sample information, plus one gene annotation file for all of them. Listing them is public, and asking for a download link needs a token. The link you get back works without one for 15 minutes by default.

# Needs jq.
BASE="https://blueprint.love"

# 1. Find the file in the public catalogue.
FILE_ID=$(curl -s "$BASE/api/download/files?project=TCGA" |
  jq -r '.data.files[] | select(.cohort == "TCGA-BRCA" and .metric == "tpm") | .id')

# 2. Ask for a link, then fetch the file. The link needs no token.
LINK=$(curl -s -X POST -H "Authorization: Bearer $BLUEPRINT_TOKEN" \
  "$BASE/api/download/files/$FILE_ID/link")
curl -o "$(echo "$LINK" | jq -r '.data.filename')" "$(echo "$LINK" | jq -r '.data.url')"

Compare the downloaded file with the catalogue's sha256. The same files are on the Downloads page.

Handle errors and rate limits

Check the HTTP status before reading the body; JSON errors carry the reason in message. A 401 from sign-in means wrong credentials; anywhere else it means refresh the token or sign in again. A 429 or 503 means wait the seconds in Retry-After and try again, which each language can do for you:

# --retry waits out 429 and 503 answers, honouring Retry-After;
# -f keeps the refused attempts' bodies out of the output.
curl --retry 5 -fsS "https://blueprint.love/api/projects?program=TCGA"

Be careful retrying job creation automatically: a plot request that timed out may have been queued anyway, so list your jobs before sending it again. Every status the API uses is in the reference.

Cite the data

The TCGA data comes from the TCGA Research Network. The National Cancer Institute asks publications to acknowledge it like this:

The results published here are in whole or part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga.

Cite the methods you ran as well, such as DESeq2, limma or Seurat, as their authors ask.