The reason economists say that taxes on land value (and pollution or congestion of a commons) is the "least bad" tax is that such taxes have no deadweight loss. Any tax on production or exchanges or movable assets causes economic inefficiency. A tax on these things causes a deadweight loss (i.e. allocative inefficiency) because people who would have more marginal benefit than marginal cost are not buying the good or service, just as a subsidy causes people to buy who impose more marginal cost than their marginal benefit. However, this effect of taxation does not happen when the supply of the taxed good is perfectly inelastic, as is the supply of land -- more precisely, the area on the surface of the Earth. Sites cannot hide, they cannot flee, and the available amount of them cannot be changed. (When a tax is not on a good but rather on a "bad", like pollution or congestion, it's the very absence of the tax that causes allocative inefficiency, because external costs are not internalized.)
"Taxing" land value (i.e. geo-rent) is less unjust than any tax on production or exchanges, because geo-rent is not created by the land-holder. Geo-rent is the extra income a site earns because of the exclusivity of its location within the community, as compared to what such a site would earn at the edge of the community. (Technically, geo-rent is the extra production you get from exclusive use of a site compared to the most productive available site that is not in use, given the same inputs of labor and capital.)
Taxing geo-rent is not only more efficient than taxing production or exchanges, but it also is less intrusive. All the government needs to know is who owns each plot of land and how much the unimproved land is worth. Appraisers and insurers make such calculations routinely, and one variant would have each land-holder self-assess as long as he's willing to take any offer over his assessed value. There's no need to audit anyone's behavior, as with taxes on income/production/exchanges. You don't even need to visit the site or look over the fence, as you do with taxes on land improvements or square footage.
Finally, taxing geo-rent imposes a built-in ceiling on government revenue. Critics of land value taxes claim they wouldn't raise enough revenue because geo-rent is allegedly only a small fraction of GDP. That sounds like a good thing to me. If government revenue is restricted by definition to geo-rent and fees for polluting/congesting/consuming the commons, then government cannot be nearly as big as when it is allowed to tax labor, production, exchanges, and all resulting products. Once you have taxation of people's labor and exchanges and produced assets, there is no limit to what the government can take from you.
Professor Foldvary lays out all these arguments in these two excellent papers:
- The Ultimate Tax Reform: Public Revenue from Land Rent - Foldvary (2006)
- Geo-Rent: A Plea to Public Economists - Foldvary (2005)