A bittersweet announcement! For family reasons, I will be leaving AISI soon to move back to the Bay Area. I will be starting a new nonprofit alignment research org (more to come). I will miss this place! Here are some reflections about my time at AISI.
Selected papers
Fine-tuning language models from human preferences
We fine-tune pretrained language models using human feedback for various tasks, successfully matching the preferences of the external human labelers, though those preferences did not always match our own. Specifically, for summarization tasks the labelers preferred sentences copied wholesale from the input (we’d only asked them to ensure accuracy), so our models learned to copy. Summarization required 60k human labels; simpler tasks which continue text in various styles required only 5k. Our motivation is to move safety techniques closer to the general task of “machines talking to humans,” which we believe is key to extracting information about human values.
AI safety needs social scientists
Properly aligning advanced AI systems with human values will require resolving many uncertainties related to the psychology of human rationality, emotion, and biases. These can only be resolved empirically through experimentation — if we want to train AI to do what humans want, we need to study humans.
AI safety via debate
We propose an AI safety technique which trains agents to debate topics with one another, using a human to judge who wins. We believe that this or a similar approach could eventually help us train AI systems to perform far more cognitively advanced tasks than humans are capable of, while remaining in line with human preferences. We outline this method together with preliminary experiments and release a web interface so people can experiment with the technique.
Deep network guided proof search
We apply deep learning based guidance to proof search in
the theorem prover E. Using strategies that leverage deep neural networks, we
have found first-order proofs of 7.36% of the first-order logic translations of
the Mizar Mathematical Library theorems that did not previously have ATP
generated proofs. This increases the ratio of statements in the corpus with ATP
generated proofs from 56% to 59%.
TensorFlow: A system for large-scale machine learning
TensorFlow is a machine learning system that operates at
large scale and in heterogeneous environments. TensorFlow uses dataflow graphs
to represent computation, shared state, and the operations that mutate that
state. It maps the nodes of a dataflow graph across many machines in a
cluster, and within a machine across multiple computational devices, including
CPUs, GPUs, and TPUs. TensorFlow supports a variety of applications, with
particularly strong support for training and inference on deep neural networks.
Several Google services use TensorFlow in production, we have released it as an
open-source project, and it has become widely used for machine learning
research.
Pentago is a first player win: strongly solving a game using parallel in-core retrograde analysis
We present a strong solution of the board game pentago,
computed using exhaustive parallel retrograde analysis in 4 hours on 98304 ($3
\times 2^{15}$) threads of NERSC’s Cray Edison. At $3.0 \times 10^{15}$
states, pentago is the largest divergent game solved to date by two orders of
magnitude, and the only example of a nontrivial divergent game solved using
retrograde analysis.
A deterministic pseudorandom perturbation scheme for arbitrary polynomial predicates
We present a symbolic perturbation scheme for black box
polynomial predicates which uses an infinite series of infinitesimal
perturbations. Our method is as fast as Emiris and Canny’s randomized linear
perturbation scheme, scaling reasonably with the degree of the polynomial even
for fully degenerate input. Like Yap’s multiple infinitesimal scheme, the
computed sign is deterministic, never requiring an algorithmic restart.
Developing fractal curves
We introduce sculpural forms which replace the resolution
dimension of L-systems with a third space dimension, turning a fractal curve
into a surface. The distances between the steps of the sequence are scaled
exponentially, so that self-similarity of the curves is reflected in
self-similarity of the surface.
Robust high-resolution cloth using parallelism, history-based collisions and accurate friction
We simulate high resolution cloth consisting of up to 2
million triangles with highly detailed folds and wrinkles. To achieve this
level of detail, we use a more accurate model for cloth-object friction, a
robust history-based repulsion/collision framework, and distributed memory
parallelism. The algorithm is demonstrated by several high-resolution and
high-fidelity simulations.
IEEE TVCG 2008
Volume conserving finite element simulations of deformable models
We model highly deformable nonlinear incompressible solids
by conserving volume locally near each node in a finite element mesh. Our
method works with arbitrary constitutive models, and works with simple linear
tetrahedra without locking. We correct errors in volume without introducing
oscillations by treating position and velocity in separate implicit solves, and
treat treat both object contact and self-contact as linear constraints during
the incompressible solve to alleviate issues with conflicting constraints.
SIGGRAPH 2007
Efficient simulation of large bodies of water by coupling two and three dimensional techniques
We simulate large bodies of water with complex surface
effects by combining tall cells with linear pressure profiles with small cells
near the interface. The philosophy is to use the best available method near
the interface (in the three-dimensional region) and to coarsen the mesh away
from the interface for efficiency. We coarsen with tall, thin cells (as opposed
to octrees or AMR), because they maintain good resolution horizontally allowing
for accurate representation of bottom topography.
SIGGRAPH 2006
TensorFlow
Eddy
Geode
Pentago
Fractal