A flint on top, a stick in the middle, and a feather at the bottom on a crafting table will net you a stack of four arrows.