Sitemap
A list of all the posts and pages found on the site. For you robots out there, there is an XML version available for digesting as well.
Pages
Posts
Improved batch norm with hijax?
Published:
Stateful operations in JAX, such as batchnorm, can sometimes be annoying to work with. As a DSL that relies on pure functions, it requires the usual functional approach of passing around a different state object to all the functions, changing $f :: x \to y$ to $f :: (x, state) \to (y, state)$. While this may not sound like much work, it can be quite tedious to manage with large and complex machine learning network workflows. Part of this complexity comes from the fact that we often want to use vmap to batch, rather than having a batch dimension for everything. Although different libraries in JAX implement stateful operations in different ways, they have a common mechanism for batchnorm, all of them keep track of a vmap named axis, then do collective operation like pmean to get the stats across the vmapped batch. Here, we won’t be looking to simplify that dimension of batchnorm, but the state management.
Numpy Random Interface in JAX
Published:
In this blog, we’ll be looking at one of the “sharp bits” of JAX: pseudorandomness, and see if we can’t smooth it out a little. JAX is an accelerated linear algebra library that forms the backbone of some modern python based scientific and machine learning libraries (things like Google’s Gemini were built on JAX). Personally I’m a fan.
Can Quantization Reduce Separability?
Published:
I was recently experimenting with the linear separability of embeddings, and showed a graph which indicated the embeddings were substantially more linearly separable than the quantized embeddings. One person commented that this didn’t make sense, and the quantized vectors should be as separable as the non-quantized vectors. That didn’t seem right to me, but on the spot I couldn’t think of a trivial counter example. In this blog, I present two.
Why The AI Bubble Will Never Pop
Published:
As the AI hype train has progressed and consumed upwards of 35% of the US economy (which surely is not a problem), a growing sentiment has emerged: that this is a bubble. This is probably now even the dominant opinion: that given the scale of investment, the large valuations, the lack of returns, and the slow progress, this constitutes a bubble. And bubbles always pop.
portfolio
Portfolio item number 1
Short description of portfolio item number 1
Portfolio item number 2
Short description of portfolio item number 2
publications
Paper Title Number 1
Published in Journal 1, 2009
This paper is about the number 1. The number 2 is left for future work.
Recommended citation: Your Name, You. (2009). "Paper Title Number 1." Journal 1. 1(1).
Download Paper | Download Slides | Download Bibtex
Paper Title Number 2
Published in Journal 1, 2010
This paper is about the number 2. The number 3 is left for future work.
Recommended citation: Your Name, You. (2010). "Paper Title Number 2." Journal 1. 1(2).
Download Paper | Download Slides
Paper Title Number 3
Published in Journal 1, 2015
This paper is about the number 3. The number 4 is left for future work.
Recommended citation: Your Name, You. (2015). "Paper Title Number 3." Journal 1. 1(3).
Download Paper | Download Slides
talks
Talk 1 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown file that can be all markdown-ified like any other post. Yay markdown!
Conference Proceeding talk 3 on Relevant Topic in Your Field
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.
teaching
Teaching experience 1
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post.
Teaching experience 2
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post.