Unfortunately that configuration is not possible within our system. One thing we absolutely cannot do is cap orders at a certain number of items for a particular product.
We can apply individual costs to specific items by using packaging costs, but that requires that the shipping scales up evenly.
For example, we can charge $4.95 per four units of a particular item which would mean that an order of 1-4 costs $4.95 shipping, an order of 5-8 would be $9.90 and so on. We could also bundle those items into a larger item's shipping cost.
Here's one way to create your settings to get something close to what you describe:
1 - Set Product 1 to ship in a Large Tube. Set the packaging capacity for Product 1 to 9999.
2 - Set Product 2 to ship in a Regular Tube. Set the packaging capacity to 4.
3 - Set Product 3 to ship in a Regular Box (so it will not get packed with items 1 or 2), set packaging capacity to 3
Next go to Seller Admin > Edit E-junkie Cart Shipping Settings and enter the following settings on the first screen:
Set Handling to 2.00 flat
Set the cost of a Large Tube to 18.00
Set the cost of a Regular Tube to 2.95
Set the cost of a Regular Box to 8.95
Press Next to save those settings, then leave the shipping rules screen since those settings aren't being used in this particular setup.
The end result of these settings would be as follows:
An order of Product 1 will cost $20 to ship, $2 for handling and $18.00 for the tube package. This cost is bundled together into a single amount for shipping on the checkout screen. Any number of Product 2 ordered alongside that will be set to fit in that same "tube" (these settings don't have to have anything to do with your actual packaging material of course).
Orders of Product 2 that do not include Product 1 will cost $4.95 to ship for quantities from 1 to 4, as there will be $2 for handling and $2.95 for the smaller package. For orders of 5-8 the shipping will go up to $7.90. Orders of 9-12 would be $10.85 and so on.
Orders of Product 3 will cost $10.95 for quantities of 1 through 3, $2 handling and $8.95 for the box. A further $8.95 will be added on for each additional box required, so orders of 4-6 would be $19.90, 7-9 would be $28.85 and so on. This setting deviates the most from what you were looking for and can't cap at 11 but who is going to order these individually when shipping on the cases of 12 is so much better? That will at least discourage large orders of loose units on this product. You can also adjust how quickly the shipping scales up for this product by lowering the package price and/or increasing the capacity.
Another thing to note is that if someone combines product 2 and 3 or 1 and 3 then handling only applies once before the separate packaging kicks in.