Participate in Experiments

Ought is a non-profit research lab building a system to delegate cognitive work on open-ended questions to human and machine learning (ML) experts. We hope that our system will eventually be able to answer questions like “Should I get LASIK surgery to correct my vision” and "What are the risks to deploying this AI system?".

Our current research focuses on amplification, an approach to AI alignment for ML-based systems. We are exploring this approach by running online experiments with human participants.

We’ve gotten to the point where we’ve made enough progress on our software and initial experiments that our next big step will be to scale up our experiments.

So, we’re looking to hire contractors ($25/hour) to participate in our experiments.

Exciting things about this opportunity:

  • The experiments can be pretty fun and interesting. Each experiment takes the form of a game. An honest expert tells the truth, a malicious expert lies, and a judge has to figure out which is which. There’s a lot of opportunity to think of strategies to fool the other participants, to practice epistemic hygiene as a judge, etc. This record of a past experiment gives a flavor of what they’re like.
  • You can contribute a lot if you can learn to play at a high level. We need participants to play really well so that we can gain confidence that the results of the experiments are due to the structure of the game, not due to mediocre performance by the players.
  • You can contribute a lot if you care about advancing our mission. There are a number of “holes” in the app and in the game rules, such that if participants just try to win at the game, the experiments won’t work. We need participants who will avoid exploiting the holes and will give us feedback to help us patch them. So, if you turn out to be an unusually skilled participant who cares about advancing our mission, you can make a large contribution to our research progress.
  • We care about how much you contribute to our research, not your qualifications. E.g. on the job application, we require you to take a quiz that’s similar to the experiments, but we don’t require you to send us your resume.

This is a pretty unique (as in, different from other opportunities) way to help out with AI safety:

  • Remote work with flexible hours - the experiment is turn-based, so you can participate at any time of day.
  • We expect that skill with language will be more important than skill with math or engineering.

Unexciting things about this opportunity:

  • This is our first time paying people to participate in our experiments. Everything about this is probably going to be pretty rough. We might have no work for you for a month, we might stop working with you for reasons that reflect no failing on your part, our systems might break down, etc.
  • Participating in experiments can be stressful and annoying. Lots of typing, lots of trying to figure out what other people mean, a fair amount of repetition because similar questions get asked over and over, lots of trying to trick people and avoid being tricked. Also other participants may take their turns at any time of day, so at any given time you may be waiting on them and have no work available.

You might be perfect for this if:

  • You’re excited about helping us move our research forward.
  • You’re skilled at reading comprehension, argument/debate, and writing, and you enjoy those things.
  • If things go well, you’d likely want to devote 5-20 hours/week to this for at least a few months. We think that participants will need to build up skill over time to play at their best, so we think it’s important that people stick around for a while.

The application takes about 20 minutes. If you pass this initial application stage, we’ll pay you the $25/hour rate for your training and work going forward.