In the product settings for the item that ships in a USPS flat-rate box, specify a shipping weight of 0.00 and select a Package Type that you will only use for that product, with a Capacity indicating how many units of that product can be shipped within that flat rate. Then in your Cart Shipping Settings, set that flat rate as the Cost of the Package Type you use for that product, but leave the package Weight at 0.00. Leaving the product and package weight sttings at 0.00 will exclude this item from any UPS/USPS rate lookups.
For the other item that ships by UPS, you could just use the same method as above to define a different nationwide flat rate, which you'd simply assign as the Cost for a different Package Type you only use for this product. However, if you prefer to have us perform a UPS Ground rate lookup for this item, be sure you specify an accurate per-unit Weight in the product's settings, select a different Package Type than your other product, and leave the empty Weight of that Package Type at 0.00 (or enter the empty weight of the container itself) in your Cart Shipping Settings. If the standard UPS Ground rate we obtain from UPS for this item is insufficient (e.g., because the item is oversize/overweight for basic UPS Ground service), you may wish to also specify a Cost for this item's Package Type that would be added as a flat surcharge on top of the base UPS Ground rate we obtain.
It appears you've deleted any shipping rules you may have previously set up, so I can't tell the reason for the shipping -error- you mentioned anymore, but that -error- would indicate the buyer's cart order does not qualify for any shipping rules you have set up.