Yes, the ZIP code method of configuration you described would be correct. Note you can use partial zip codes to cover a range -- e.g., this:
...would apply 5% tax to ZIPs 98700 thru 98799, except for 98760-98769 at 5.5% and 98765 at 6%.
Supporting configurations by county would be, unfortunately, both complicated to implement and unreliable in practice. Aside from Admin settings, we'd need to modify the cart itself to add a field where buyers could enter their county, then what if they don't know what county they're in or misspell it? Finally, county information is never on record nor requested on third-party payment processor (e.g. PayPal) checkout pages, so there'd be no way to catch a mismatch between the county the buyer entered in the cart vs. their actual billing/shipping address.
Our Development team is aware of the situation with many states adopting location-specific tax rates lately, so a better way to handle that type of taxation is on the wishlist and I think underway as an eventual new feature.