EDHREC at Home: Talking to Scryfall

Interfacing With Your Boss

Previously on The Card Recommender, I created a model for a Magic card and defined it in code. I also scoped out a place where I can find all the information I need to know about the game objects my system will recommend: Scryfall. I need to teach my recommender how to fetch the data it needs to run; it’s time to put the pedal to the road, and make the rubber meet the metal.

Computers are Dumb

Before we begin, here’s a quick review of some vocabulary from last time. An API, or application programming interface, refers both to a specification that computers use to interact and to any program that implements such a specification. It’s like how a restaurant menu lets you know what you can order.

My plan is to use Scryfall’s API to download information about every card in Magic.

Now, the above definition for the term API is accurate, but it’s also quite broad and abstract. A “specification computers use to interact” could mean almost anything! I need to know concrete details to get any work done.

Computers, after all, are machines, and machines can’t think, reason, or solve problems on their own, as much as certain companies are trying to trick you into thinking otherwise.

No, computers can only do what a programmer explicitly tells them to. Of course, telling computers what to do is a lot of hard work — but there’s one thing about engineering that makes my life so much easier: human beings are super, duper lazy — and while we’re on the topic, I have a point to make.

On Laziness

In the 1987 film Wall Street, Gordon Gekko is a cutthroat financier who leverages insider information to illegally make a killing in the stock market.1

Partway through the movie, during a speech at a stockholder meeting, Gordon extolls the supposed virtues of avarice as he recounts his successes. To him, the financial incentive works as an optimizing force, which shaves away needless inefficiencies and encourages mankind toward progress. In his words:

“Greed, for lack of a better word, is good.”

This quote is famous for two reasons. The first is that it’s shockingly counterintuitive; it takes an aspect of humanity with a universally negative connotation, greed, and reinterprets it as something venerable. If there’s anything the rise of conspiracy theories these past few years have taught me, it’s that there’s a dangerous power in the honeyed thought that the conventional wisdom is wrong. It’s tempting to lord over others by convincing yourself that everybody else is a blind sheep, and only the smart, intelligent chosen few (like yourself) know the real way the world works.

The second (and more relevant) reason for its fame is that there is logic in his argument. After all, the idea that the market knows what it wants and will push people to do their best is just called capitalism, and a hell of a lot of people believe in that.2

Now, I’m not going to sit here and defend greed — it’s not a virtue. Especially if it’s unchecked; I’ll just vaguely gesture outside to prove that point. However, the underlying idea, that our darker desires have a reason to exist and can be beneficial or even good if leveraged properly, is (on some level) fundamentally true. Right now, greed isn’t the sin I’m choosing to defend; sloth is.

Our brains are incredible, wondrous, awe-inspiring couch potatoes. They want to keep us safe, have us reproduce, and attain self-actualization. They’re also biological machines that run on a ton of precious, precious calories, so they want to do all of it with the minimum amount of effort possible. Ideally, they’re also quick enough so that a lion won’t eat you before you’re able to make a decision. Our brains evolved to take shortcuts, apply heuristics, and jump to conclusions. Our brains evolved to be lazy.

To be clear, laziness is not a virtue. In excess, sloth leads to stagnation and decay, the way an excess of greed leads to duplicity and inequality. In moderation, however, laziness is the perfect incentive for ingenuity. The reason we, as human beings, have been able to so effectively conquer challenges is that we can recognize patterns in the problems we face, create general, reusable solutions for them, and then synthesize that collective knowledge into novel ideas with which we can scale ever greater heights.

We want the shit we figure out to stay figured out, and if we have to deal with new shit, we reach in the pantry for something we’ve already prepared and change it a little with a new spice. We’re not in the business of reinventing the wheel; we’re in the business of putting four of them together with an engine to invent the automobile.

Back to the API

What this means in terms of software engineers is that, since we’re all lazy, we don’t want to come up with new ways to do things if the old ways are good enough. We also don’t really want to spend time explaining the old ways, because we’re busy and we’d rather have other people do that for us. This, of course, leads to a proliferation of standards. Sweet, sweet, cataloged-and-identified-by-8-digit-ISSN-code standards.

Last time, I linked to the Scryfall API representation for Redwood Treefolk. To you and I, it looks like a bunch of text containing information about the card, but to a computer, this file is nothing more than a long chain of zeroes and ones. In fact, that’s true of all data on a computer. Your photos are zeroes and ones, your applications are zeroes and ones, and even those messages from that cute boy calling your rump pleasant are zeroes and ones. There’s no fundamental law distinguishing one stream of binary digits as an image and another stream of binary digits as a music file.

No, our computers know how to show images and play music files because we’ve all gotten together and made programs that interpret certain patterns of zeroes and ones as pictures or songs — and then distributed our pictures and songs in those agreed-upon patterns. You’ve definitely heard of these standards before; they’re called file formats, like .jpg or .mp3. As weird as it is to think about, at the machine level, both are just really big numbers. Ever try to open up a .jpg in Notepad only to see weird gibberish? That’s because the data is being interpreted incorrectly; it would be like cracking an English novel and attempting to read it in perfect Spanish.

The first level of laziness helping me is that people have gotten together and agreed on ways to represent text in zeroes and ones. An interpretation of binary that results in text is called a character encoding. Perhaps the most famous example of a character encoding is ASCII. ASCII reads every seven bits in the file as a number, and then maps that number to a letter defined by a preset table. Scryfall will tell me what encoding it uses, so I don’t have to worry about the text getting mangled.

Then, there’s a second text standard I’m using on top of the character encoding: the text Scryfall sends back conforms to the JavaScript Object Notation standard. Also known as JSON, It’s a human readable way to arrange data so that logically related information is grouped together, and information which isn’t is explicitly separated by punctuation like brackets and parenthesis. I can now count on some additional structure in API responses, which makes it easier to tell my computer how to find and focus on the specific card facts I actually care about.

Let’s not forget the standard defining the way I’m interacting with Scryfall: Hypertext Transfer Protocol, good ol’ fashioned HTTP. It’s the same protocol that helped your browser transmit the very article you’re reading now.3

When you take all this together, Scryfall’s API isn’t actually bespoke or novel. It’s essentially just a series of web pages that I can request which returns data in a standardized, structured format. This combination of attributes is so common that there’s even a name for it: Scryfall is a REST API.4

This standardization is great news for me. The fact that Scryfall’s API is so banal means that it’s actually pretty easy to do what I want to do, because many engineers before me have had similar problems and invented general, reusable solutions that I can pull right off the shelf. Don’t believe me? Check this out.

The Bulk Data

My first step is to understand what it is I’m downloading, and for that, it’s time to go to the documentation. The Bulk Data API serves a JSON object that contains a catalog of the catalogs of cards that Scryfall publishes. Yes, there are multiple versions of “every card on Scryfall”, based on whether you want to know about all unique printings, all unique printings for all languages, or merely every unique game object. Since I only care about cards as game objects, the primary Bulk Data file I’m using will be the Oracle Cards one.

{
    "object": "bulk_data",
    "id": "27bf3214-1271-490b-bdfe-c0be6c23d02e",
    "type": "oracle_cards",
    "updated_at": "2025-07-14T21:08:01.240+00:00",
    "uri": "https://api.scryfall.com/bulk-data/27bf3214-1271-490b-bdfe-c0be6c23d02e",
    "name": "Oracle Cards",
    "description": "A JSON file containing one Scryfall card object for each Oracle ID on Scryfall. The chosen sets for the cards are an attempt to return the most up-to-date recognizable version of the card.",
    "size": 161426233,
    "download_uri": "https://data.scryfall.io/oracle-cards/oracle-cards-20250714210801.json",
    "content_type": "application/json",
    "content_encoding": "gzip"
  }

The plan for me is clear; I need to find the object with the oracle_cards type and follow its download URI to get the file I actually care about.

I’m writing my program in the C# programming language. Not only does it have a well-developed standard kit that will be instrumental going forward, but it’s also the language I’m most familiar with, and since I use it in my professional life as well as my personal life, I can apply everything I learn here to my job.5

There’s one more big plus to using C#: both JavaScript (the progenitor of JSON) and C# are object-oriented programming languages. Object-oriented programming languages represent data as collections of fields called objects; if you wanted to, for example, represent a person’s demographics, you could create a Person class and fill it with fields like FirstName, LastName, and DateOfBirth. A class is a blueprint that the computer uses to create instances of objects, and I will be writing lots of them.

Classes can also contain methods, which are snippets of code that can access the data in the object instance. I could implement a CalculateAge(DateTime time) method on the Person class, which uses a person’s date of birth and the given time to calculate what age they would be at that time. This style of programming lets you easily organize how data is laid out and operated on without needing to keep all the details of its implementation in your head, which is great for reasoning and collaboration.

To connect with Scryfall using C#, I need to write some glue code that bridges Scryfall’s public API and my own private project. Since both C# and JSON work with objects, it’s easy to translate one to the other, but it’s still a laborious process where I read the public documentation, manually convert it into C#, and repeat for every endpoint I need to use. Here’s what the end result looks like. Feel free to compare my BulkData class to the example JSON object above.

public class BulkData
{
    [JsonPropertyName("id")]
    public required string Id { get; set; }

    [JsonPropertyName("type")]
    public required BulkDataType DataType { get; set; }

    [JsonPropertyName("updated_at")]
    public required DateTimeOffset UpdatedAt { get; set; }

    [JsonPropertyName("name")]
    public required string Name { get; set; }

    [JsonPropertyName("description")]
    public required string Description { get; set; }

    [JsonPropertyName("size")]
    public required long Size { get; set; }

    [JsonPropertyName("download_uri")]
    public required Uri DownloadUri { get; set; }

    [JsonPropertyName("content_type")]
    public required string ContentType { get; set; }

    [JsonPropertyName("content_encoding")]
    public required string ContentEncoding { get; set; }
}

public enum BulkDataType
{
    [JsonStringEnumMemberName("oracle_cards")]
    OracleCards,

    [JsonStringEnumMemberName("unique_artwork")]
    UniqueArtwork,

    [JsonStringEnumMemberName("default_cards")]
    DefaultCards,

    [JsonStringEnumMemberName("all_cards")]
    AllCards,

    [JsonStringEnumMemberName("rulings")]
    Rulings,
}

If you read the documentation, you’ll notice that we don’t merely receive a straight-up list of BulkData objects when we call the Bulk Data endpoint. There’s actually a wrapper that contains the data, and that wrapper is shared between many different endpoints in the Scryfall API. I’ll also represent that in my code:

public class ResultPage<T>
{
    [JsonPropertyName("has_more")]
    public bool HasMore { get; set; }

    [JsonPropertyName("next_page")]
    public Uri? NextPage { get; set; }

    [JsonPropertyName("data")]
    public List<T> Data { get; set; } = [];
}

The class ResultPage<T> is a bit special; it’s what’s known as a generic class. That <T> on the end is a type parameter, or a placeholder for the name of another type that I define when I create an instance of the object. They’re useful when you need to store or add additional information with other types, but you don’t really care about what that underlying type is.

If I were to create a Person, it would look like var sally = new Person(); if I were to create a ResultPage that contains Persons, I would do var personResultPage = new ResultPage<Person>(). The type of Data on personResultPage is a List of Person (since List<T> is itself a generic class!), but I still have the same HasMore flag and NextPage URI. With generic classes, I can reuse the same wrapper object for every endpoint in the Scryfall API without rewriting it every single time.

How Do I Download Web?

Now, to create the code that actually downloads the data. There’s a class called HttpClient in C#, which creates objects that I can use to download pages from the Internet.

In theory, this would be fine for my purposes, but in practice I’d have to translate my objects into a JSON request, set the request body, populate HTTP headers, get the URL right, manually parse the response, and more I’m probably forgetting.

That sounds complicated, so I’m not gonna do it. I present to you, unabridged, all the code I hade to write to download the Bulk Data endpoint:

using Refit;

namespace Celani.Magic.Scryfall;

public interface IScryfallApi
{
    [Get("/bulk-data")]
    Task<ResultPage<BulkData>> GetBulkDataAsync();
}

You’ll notice that there’s an astounding lack of code in this snippet. How does it work?

Well, let’s start with the first line, using Refit. Refit is not a standard part of C#; it’s an external package I downloaded from the Internet and installed in my own project.

Refit creates wrapper objects that interact with REST APIs automatically. All I have to do is define an interface with the endpoints I care about (see: [Get("/bulk-data")]) , and then I can use a (surprise!) generic method with a type parameter to create an object which reads that definition and does all the downloading for me like this:

var scryfallApi = RestService.For<IScryfallApi>("https://api.scryfall.com");
var bulkData = await scryfallApi.GetBulkDataAsync();

The real code is a little bit more complicated than that — I’m skipping over some more advanced topics for the sake of a general audience — but the end result is that I legitimately do not do anything more complicated than call GetBulkDataAsync() to get the Bulk Data Object.

The fact that Refit exists is a testament to the power of standardization. Look at all the work I didn’t have to do because someone else already did it for me! This never stops being useful; it pays dividends when I eventually need to use more endpoints from Scryfall, and it’s also instrumental when I interface with Archidekt and Moxfield.

There’s one last thing I have to do, and that’s actually download the file listed in the BulkData’s DownloadUri. That code actually does end up using something similar to the HttpClient class:

// Create an object that can download files from the Internet.
using var client = ClientFactory.CreateClient("BulkData");

// Find the Bulk Data object with the oracle_cards type.
var oracleCards = bulkData.Data.First(x => x.DataType == BulkDataType.OracleCards);

// Open a connection to Scryfall at the OracleCards DownloadUri.
using var oracleStream = await client.GetStreamAsync(oracleCards.DownloadUri);

// Download the data and turn it into a list of ScryfallCardObject objects.
var oracleCardEnum = JsonSerializer.DeserializeAsyncEnumerable<ScryfallCardObject>(oracleStream, BulkDataDownloadOptions);

The ScryfallCardObject is another glue class, adapted from the API’s definition of a card. Its implementation details are not especially important right now, but it’s worth pointing out that this is a separate and distinct class to the OracleCard class I created last time. I have to create two classes, because ScryfallCardObject is mirroring what Scryfall provides me, and I’m at their mercy when it comes to interfacing with them.

Next Time

We’ve interrogated Scryfall’s REST API and actually downloaded some data! Now we have to store it — and that’s a topic for another day. ‘Til then!

Footnotes

  1. I swear I’m going somewhere with this. ↩

  2. As an aside, the reason Gekko is such a compelling villain is that he uses his philosophy to rationalize his illicit and immoral actions; the movie examines and ultimately rejects those ideas thorough the eyes of its main character, Bud Fox, who falters under the sway of Gekko’s tutelage before ultimately doing the right thing and turning himself in. ↩

  3. If you thought it might be stupid to even think about this as a potential problem, some APIs work by directly connecting to another computer to send raw streams of bytes back and forth! ↩

  4. Well, strictly speaking, a REST API has more design and architectural constraints, but a lot of entities including Scryfall themselves use the term to refer to HTTP-based web APIs as a whole and arguing with it is pedantic so let’s move on. ↩

  5. In theory, improving my skillset should earn me more money, but in practice my only reward will be more work. ↩