Rest api for google sheets

[RickiHeicklen]

I'd like to be able to use Google Sheets. I know this doesn't feel top priority but I am unsatisfied with the current state of scenarios monitoring, and I want to be able to just use Google Sheets, so I'm doing it.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
platform	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 11, 2025 9:03pm

[Crazytieguy]

Overall great job, this certainly accomplishes what we need. I've left a bunch of low level comments on the code, with the only important one being that currently if you don't pass an authorization header you can just get admin access. Some high level discussion + comments:

General approach

Setting up a proxy API server was certainly a viable decision, but it's not the only one. The most obvious alternative is to add these endpoints to the existing rust server, which happens to work similar to flask except the one endpoint it currently has happens to be a websocket endpoint, allowing a persistent bi-directional channel with the client. The implementation I had in mind as taking 1-2 days was to embed javascript on the google sheet using a custom menu instead of appsscript, which would allow to create a regular connection to the server like the frontend does and update the google sheet live. I think in the future we'll want to transition to the live updating version, at which point we can decide if to keep the rest api for other purposes.

Tech choices

• uv for dependency management would have been nice, it solves a bunch of problems and is already being used in the codebase.
• Flask is kind of legacy at this point IMO, and I don't like it very much. I get why an llm recommended it though, it's was the standard for python api servers for a long time. I prefer FastAPI, which is the library we're using for scenarios that auto-generates the nice interactive API docs.
• Overall a pretty easy way to set this up would have been to copy over the stuff in the scenarios repo and delete the anything that's not needed.

Obscure fears

I'm a bit concerned that if we give the students a refresh button on the spreadsheet we'll hit the appsscript rate limit and everything will stop working for everyone, so maybe we should be cautious about that somehow. Also I have a suspicion that with all the students having cached clients we'll run out of the 512mb of memory once there are bots involved.

LMK if you want me to elaborate / dive deeper on anything!

[Crazytieguy]

This is a non standard header format, which might make it weird for people to try using the rest api not through the google sheet. I'd probably put the act as in a query parameters (it can be left out - act_as is allowed to just be 0 signifying 'act as myself'), and the standard format for the authorization header is Bearer <jwt>, which flask probably has a way of parsing for you.

[Crazytieguy]

Actually, I think you probably just don't want them to pass act_as at all since they're not sending orders anyway. They'll get visibility of all of their owned (recursive) accounts whether they act_as or not

[RickiHeicklen]

let me know if the current one is better?

[Crazytieguy]

As a general rule you want to return an object (dict) as the top level of the response rather than an array. This is more self describing, and it lets you add more fields to the response later without breaking existing clients. So something like (pseudo code):

{
  trades: [
    ...
  ]
}

[RickiHeicklen]

I made a change that just takes the trades directly and JSONify's them directly
trades = state.markets[market_id].trades
Is this fine? It ends up outputting things like this (this one is for orders but you can image the same concept for trades):

If not, I think I don't fully understand the syntax, can you explain again? Is something being assigned as a valuable to that object? I'm getting issues about trades being unbound

[Crazytieguy]

This output looks fine, to understand the issue you're describing I'd have to see the code

[RickiHeicklen]

FYI, this is where the spreadsheet currently lives: https://docs.google.com/spreadsheets/d/1QQ7GVo9KDLbJduu7iQZwSSDVAOTNCJBrsRLTNlw0SKc/edit?gid=0#gid=0

And this is the code I have in the apps script:

function getJWT() {
  try {
    const jwt = SpreadsheetApp.getActiveSheet().getRange('J3').getValue();
    if (!jwt) {
      throw new Error('JWT is not set. Please add your JWT to cell A1.');
    }
    return jwt;
  } catch (error) {
    throw new Error('Could not read JWT from cell A1. Please ensure your JWT is in cell A1.');
  }
}

function callMiddlewareAPI(endpoint, errorMessage) {
  const jwt = getJWT();
  
  const options = {
    headers: {
      'Authorization': `Bearer ${jwt}`
    },
    muteHttpExceptions: true
  };

  try {
    const response = UrlFetchApp.fetch(`https://middleware-api.fly.dev${endpoint}`, options);
    const responseCode = response.getResponseCode();
    
    if (responseCode !== 200) {
      throw new Error(`HTTP ${responseCode}: ${response.getContentText()}`);
    }
    
    const contentText = response.getContentText();
    if (!contentText) {
      throw new Error('Empty response from server');
    }

    let data;
    try {
      data = JSON.parse(contentText);
    } catch (e) {
      throw new Error(`Invalid JSON response: ${contentText}`);
    }

    if (data === null) {
      throw new Error('Null response from server');
    }
    
    if (data.error) {
      throw new Error(data.error);
    }
    
    return data;
  } catch (error) {
    console.error(errorMessage, error);
    throw new Error(`${errorMessage} ${error.message}`);
  }
}

function getAvailableBalance() {
  const data = callMiddlewareAPI('/portfolio', 'Error fetching portfolio:');
  return data.available_balance;
}

function getBid(marketId, depth) {
  const data = callMiddlewareAPI(`/market/${marketId}/bid/${depth}`, 'Error fetching bid:');
  if (!data || !data.price) {
    throw new Error('No bid found at specified depth');
  }
  return data.price;
}

function getBidSize(marketId, depth) {
  const data = callMiddlewareAPI(`/market/${marketId}/bid/${depth}`, 'Error fetching bid:');
  if (!data || !data.price) {
    throw new Error('No bid found at specified depth');
  }
  return data.size;
}

function getOffer(marketId, depth) {
  const data = callMiddlewareAPI(`/market/${marketId}/offer/${depth}`, 'Error fetching offer:');
  if (!data || !data.price) {
    throw new Error('No offer found at specified depth');
  }
  return data.price;
}

function getOfferSize(marketId, depth) {
  const data = callMiddlewareAPI(`/market/${marketId}/offer/${depth}`, 'Error fetching offer:');
  if (!data || !data.price) {
    throw new Error('No offer found at specified depth');
  }
  return data.size;
}

function midprice(marketId) {
  const data = callMiddlewareAPI(`/market/${marketId}/mid`, 'Error fetching midprice:');
  if (!data.midprice) {
    throw new Error('Unable to calculate midprice');
  }
  return data.midprice;
}

[RickiHeicklen]

@Crazytieguy I think I addressed most of your comments. The things that didn't work the way I'd hoped:

1. adding metagame to the requirements.txt file doesn't seem to allow me to delete the importation from github (i.e. the line RUN pip install git+https://github.com/tradingbootcamp/platform.git#subdirectory=python-client)
2. I can't get uv working correctly - but I gave up pretty quickly. Happy for you to make and me to review the changes. It's possible the problem is I'm not running some necessary command?

[Crazytieguy]

Small apps script review

function getJWT() {
  try {
    const jwt = SpreadsheetApp.getActiveSheet().getRange('J3').getValue();

I think you're reading from J3, but asking to write to A1? Also you can give the cell a name and then refer to it by name.

    if (!jwt) {
      throw new Error('JWT is not set. Please add your JWT to cell A1.');
    }
    return jwt;
  } catch (error) {
    throw new Error('Could not read JWT from cell A1. Please ensure your JWT is in cell A1.');
  }
}

It looks like you're catching the thrown exception just to immediately throw a basically identical exception. I think you don't need the try / catch at all.

The rest seems good I think

[Crazytieguy]

I made the switch to uv, but I wasn't able to verify that it works correctly because the docker registry is having a production incident (rare!). I think you accidentally deleted the first line in the pyproject.toml file and maybe that caused you problems

[Crazytieguy]

I also renamed middleware to rest proxy (but I left the fly app name since you've already deployed it). BTW you deployed it to your personal fly.io I think, so if you want anyone else to have access to the deployment you'll have to add us as collaborators to your fly account, or you can delete that deployment and we can deploy it from my account

[RickiHeicklen]

Hm, I tried the deleting and redeploying from the organization approach, and something seems to not be working. I'm not entirely sure how to best approach testing it. Let me know if you have free time at some point to walk me through that process?

[Crazytieguy]

I tested the dockerfile and fixed it, if that was your issue it should be good now. Otherwise I'm available rn, you can catch me on discord