Make a short spacer/washer with the outside diameter of the spindle shaft and the inside diameter of the bolt, and thick enough to allow you to tighten the bolt without binding the bearing.
You will still use the larger washer under the head of the bolt to keep the wheel on the shaft.
BTDT