Security startup Strix was looking for an AI inferencing offering and found Baseten. Out of caution, the former scanned the latter for any potential vulnerabilities. Within 25 minutes, a live GitHub token with admin rights appeared. The token itself was already over 3 years old.
Strix followed its company policy in scanning prospective vendors before handing over their data, models or code. The team aimed its security agent at *.baseten.co without any additional aid or context. A personal access token belonging to the ‘basetenbot’ account appeared readily. Alex Schapiro, co-founder of Strix, notes that the token carried admin and push rights on Baseten’s product repository, the GitOps repo driving its clusters as well as its Homebrew tap. Beyond that, read/write access to several private repositories, including customer-specific ones, were found.
Baseten is an AI neocloud, an infrastructure provider for inferencing workloads akin to the likes of CoreWeave, Nebius and Nscale. A recent report from SemiAnalysis determined that most such neoclouds “suck at security”, leaving plenty of reason for potential customers to follow Strix’s lead in approaching cautiously before consdering handing over data. Baseten, valued at 13 billion dollars and active in multiple regions, fell foul of this check.
From public registry to build history
The agent Strix deployed started by enumerating hosts, checking certificate logs and mapping the attack service. It eventually landed on a Harbor container registry on a Baseten subdomain. Following on from this reconnaisance step, the agent found a public project which allowed anonymous pull tokens and downloads of image manifests and blobs.
A first pair of AWS keys inside the baseten-app image turned out to be dead. As the agent kept digging, however, it ran TruffleHog across the layers, and then inspected the image config itself. The token sat in the history[].created_by field, where Docker records how a build step was created. In this case, a RUN command with the value of GITHUB_TOKEN expanded into it was found. Strix used it for a read-only request and, as Schapiro phrases it, “VOILÀ.”
Baseten rotated the token within a day
Schapiro reported the finding on July 13th; the next morning, Baseten took action. It made the Harbor project private, confirmed the rest of the issues and rotated its token. By July 17th, all flagged security flaws were resolved.
The neocloud was ultimately lucky that a potential customer effectively performed an unwitting courtesy redteaming operation. Should such neoclouds become more prevalent for critical organizations, expect more sloppy practices to be exploited, and for these infrastructure providers to learn what it takes to offer secure cloud services. At least Baseten course-corrected quickly, which is a good sign.