API documentation
Walkthroughs for the tasks scripts usually need, each with curl, Python and R versions that run against this site.
Before you start
The samples in these guides assume three things:
- The tools are installed:
curlandjqfor the shell,pip install requestsfor Python,install.packages("httr2")for R. - Samples that need a token read it from
BLUEPRINT_TOKEN. Set it as in the quickstart, or let your script sign in by itself. - Samples that sign in read
BLUEPRINT_USERandBLUEPRINT_PASSWORD. Set them without leaving the password in your shell history or a notebook:
printf 'Username: '; read -r BLUEPRINT_USER
printf 'Password: '; read -rs BLUEPRINT_PASSWORD; echo
export BLUEPRINT_USER BLUEPRINT_PASSWORDInput is hidden where it says so; press Enter to finish each line.
Sign in from a script
A token copied from your Profile page lasts 15 minutes by default. For anything longer, let the script sign in with your username and password, keep the refresh cookie, and refresh when a call answers 401.
Create the account on the website first; registration needs a code sent to your email address or phone. The sample reads BLUEPRINT_USER and BLUEPRINT_PASSWORD, set as in Before you start.
# Needs jq. The cookie jar holds the refresh token: it is a credential,
# so it lives in a private temporary file (mktemp creates it mode 0600).
BASE="https://blueprint.love"
COOKIES=$(mktemp)
login() {
body=$(jq -n --arg u "$BLUEPRINT_USER" --arg p "$BLUEPRINT_PASSWORD" \
'{username: $u, password: $p}' |
curl -s -c "$COOKIES" -H "Content-Type: application/json" -d @- \
"$BASE/api/auth/login")
if echo "$body" | jq -e '.data.requires_2fa' >/dev/null; then
echo "Two-factor account: finish with POST /api/auth/2fa/verify" >&2
return 1
else
# Prints nothing, and says why, unless the sign-in worked.
echo "$body" | jq -r '.data.access_token // ("Sign-in failed: " + .message + "\n" | halt_error)'
fi
}
refresh() {
curl -s -b "$COOKIES" -c "$COOKIES" -X POST "$BASE/api/auth/refresh" |
jq -r '.data.access_token // ("Refresh failed: " + .message + "\n" | halt_error)'
}
BLUEPRINT_TOKEN=$(login) &&
curl -s -H "Authorization: Bearer $BLUEPRINT_TOKEN" "$BASE/api/auth/me" |
jq -r '.data.username'
# Later, when a call answers 401:
BLUEPRINT_TOKEN=$(refresh)- Sign-in allows 3 per second per IP address, and 10 failed attempts for one username or email address, each within 30 minutes of the one before, lock sign-in with it for 15 minutes. Sign in once and refresh; do not sign in before every call. A
401from sign-in means wrong credentials: refreshing will not help. - Each refresh replaces the cookie. If two refreshes race, one answers
409: retry it. - With two-factor authentication on, the Python sample asks for your code; the curl and R samples stop and say so.
- Treat the password and tokens as secrets. If a token leaks, sign out on the website: that ends every session and revokes the tokens issued before it, including the script's. If your password leaks, change it in Settings, which also ends every session.
Page through projects and samples
GET /api/projects is public. Ask for pages with page and pageSize, and stop at totalPages. Filter with program, primary_site or disease_type; the accepted values come from GET /api/projects/filters.
A project's samples, with clinical fields such as vital status and follow-up days, need a token and come at most 500 to a page.
# Needs jq.
BASE="https://blueprint.love"
page=1
while body=$(curl -fsS "$BASE/api/projects?program=TCGA&page=$page&pageSize=20"); do
echo "$body" | jq -r '.data.items[] | [.id, .description, .case_count] | @tsv'
[ "$page" -ge "$(echo "$body" | jq '.data.totalPages')" ] && break
page=$((page + 1))
done # -f stops the loop on an HTTP error instead of paging on forever
# Samples need a token, and a page holds at most 500.
curl -s -H "Authorization: Bearer $BLUEPRINT_TOKEN" \
"$BASE/api/projects/TCGA-BRCA/samples?page=1&pageSize=100" |
jq -r '.data.items[] | [.barcode, .sample_type, .vital_status] | @tsv'A project's case_count counts samples, and one patient can give more than one, so it is not the number of patients.
Render a figure
Figures render as jobs. This one draws TP53 expression across TCGA-BRCA tissue types as a box plot, waits for it, and saves the PNG.
# Needs jq.
BASE="https://blueprint.love"
AUTH="Authorization: Bearer $BLUEPRINT_TOKEN"
RESP=$(curl -s -H "$AUTH" -H "Content-Type: application/json" -d '{
"plotType": "boxplot", "projectId": "TCGA-BRCA", "geneId": "TP53",
"x": "tissue_type", "y": "vstd",
"colorScheme": "tab10", "width": 800, "height": 600, "format": "png"
}' "$BASE/api/plot/jobs")
JOB=$(echo "$RESP" | jq -r '.data.id // empty')
[ -z "$JOB" ] && echo "$RESP" | jq -r '.message' # refused: says why
STATUS=
while [ -n "$JOB" ]; do
STATUS=$(curl -s -H "$AUTH" "$BASE/api/plot/jobs/$JOB" | jq -r '.data.status')
case "$STATUS" in queued|processing) sleep 2 ;; *) break ;; esac
done
if [ "$STATUS" = completed ]; then
# -f: a 404 here means the figure is still being stored; run this line again.
curl -fsS -H "$AUTH" -o tp53-brca.png "$BASE/api/plot/jobs/$JOB/result"
elif [ -n "$JOB" ]; then
curl -s -H "$AUTH" "$BASE/api/plot/jobs/$JOB" | jq -r '.data.error_message'
fi- Poll no more often than every 2 seconds. A failed job's
error_messagesays why. - Submissions allow 30 per minute, and you can hold 20 unfinished plot jobs by default.
formatalso takessvgandpdffor figures you will edit or print.
Run an analysis and export its results
This runs a PCA on TCGA-BRCA's variance-stabilised expression, prints the explained variance, and saves the sample scores as CSV.
# Needs jq.
BASE="https://blueprint.love"
AUTH="Authorization: Bearer $BLUEPRINT_TOKEN"
RESP=$(curl -s -H "$AUTH" -H "Content-Type: application/json" -d '{
"analysisType": "pca", "projectId": "TCGA-BRCA", "expressionType": "vstd",
"pca": {"nComponents": 5, "nTopGenes": 2000}
}' "$BASE/api/analysis/jobs")
JOB=$(echo "$RESP" | jq -r '.data.id // empty')
[ -z "$JOB" ] && echo "$RESP" | jq -r '.message' # refused: says why
STATUS=
while [ -n "$JOB" ]; do
STATUS=$(curl -s -H "$AUTH" "$BASE/api/analysis/jobs/$JOB" | jq -r '.data.status')
case "$STATUS" in queued|processing) sleep 5 ;; *) break ;; esac
done
if [ "$STATUS" = completed ]; then
curl -s -H "$AUTH" "$BASE/api/analysis/jobs/$JOB?include=result" |
jq '.data.result.explainedVarianceRatio'
curl -s -H "$AUTH" -o pca-scores.csv \
"$BASE/api/analysis/jobs/$JOB/export/scores?format=csv"
elif [ -n "$JOB" ]; then
curl -s -H "$AUTH" "$BASE/api/analysis/jobs/$JOB" | jq -r '.data.error_message'
fi- Differential expression (
deseq2,limma,limma_voom,edger) runs the same way; download its table from/api/analysis/jobs/JOB_ID/results?format=tsv. - Sending the same request again, while your earlier job is queued, running or completed, returns that job with
data.reusedset to true. After a failed or cancelled job it runs again. - Submissions allow 10 per minute.
Analyse your own data
Upload a table and use the fileId it returns wherever a request takes projectId. Send a valid token with the upload. A missing, expired or revoked token is not refused: the file is stored anonymously and your jobs cannot use it, so refresh first if the token may have lapsed.
# Needs jq.
curl -s -H "Authorization: Bearer $BLUEPRINT_TOKEN" \
-F "file=@expression.csv" "https://blueprint.love/api/upload" | jq -r '.data.fileId'- Files are limited to 50 MiB by default. Uploading the same file again returns the same
fileId. - Use delimited text such as CSV or TSV, laid out the way the figure expects. Each plot form on the Analysis pages shows that layout, and the catalogue plots publish example files under
/plot-samples/.
Download published TCGA files
The catalogue lists each cohort's count, TPM and FPKM matrices and its sample information, plus one gene annotation file for all of them. Listing them is public, and asking for a download link needs a token. The link you get back works without one for 15 minutes by default.
# Needs jq.
BASE="https://blueprint.love"
# 1. Find the file in the public catalogue.
FILE_ID=$(curl -s "$BASE/api/download/files?project=TCGA" |
jq -r '.data.files[] | select(.cohort == "TCGA-BRCA" and .metric == "tpm") | .id')
# 2. Ask for a link, then fetch the file. The link needs no token.
LINK=$(curl -s -X POST -H "Authorization: Bearer $BLUEPRINT_TOKEN" \
"$BASE/api/download/files/$FILE_ID/link")
curl -o "$(echo "$LINK" | jq -r '.data.filename')" "$(echo "$LINK" | jq -r '.data.url')"Compare the downloaded file with the catalogue's sha256. The same files are on the Downloads page.
Handle errors and rate limits
Check the HTTP status before reading the body; JSON errors carry the reason in message. A 401 from sign-in means wrong credentials; anywhere else it means refresh the token or sign in again. A 429 or 503 means wait the seconds in Retry-After and try again, which each language can do for you:
# --retry waits out 429 and 503 answers, honouring Retry-After;
# -f keeps the refused attempts' bodies out of the output.
curl --retry 5 -fsS "https://blueprint.love/api/projects?program=TCGA"Be careful retrying job creation automatically: a plot request that timed out may have been queued anyway, so list your jobs before sending it again. Every status the API uses is in the reference.
Cite the data
The TCGA data comes from the TCGA Research Network. The National Cancer Institute asks publications to acknowledge it like this:
The results published here are in whole or part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga.
Cite the methods you ran as well, such as DESeq2, limma or Seurat, as their authors ask.