In a mixing bowl, combine the cold milk and the pudding mixes, whisking together for two minutes. This will now be very thick. Whisk in the peanut butter and vanilla. Fold in about three quarters of the Cool Whip, or you can use all of it depending on how fluffy you want the texture.
Unwrap the REESES MINIATURES. Save some whole for decoration.
Cut the remaining cups in half with a sharp knife.
Use a large deep serving bowl. Layer half of the crushed cookies and half of the cut up peanut butter cups, then half of the pudding mixture, then the rest of the crushed cookies and then the rest of the cut up peanut butter cups and last the rest of the pudding mixture. At this point you can put more Cool Whip on top or just decorate the top of the last layer of pudding mixture with the whole REESES MINATURE CUPS.
Refrigerate until ready to serve. Return leftovers to the refrigerator to store.