In a medium saucepan over medium heat, combine sugar, butter and milk. Bring to a rapid boil and boil for one full minute. Remove from heat and stir in the vanilla and peanut butter. Mix sauce into the oats, stirring until well mixed. Grease 9x13 pan. Pour mix into pan. Refridgerate until set. Cut into approx. 1 1/2 inch squares. Store covered.