Libertarianism and Children

Children and children's rights under a libertarian system have been the most troubling theoretical problem for me over the last couple years. No libertarian theorist has addressed this issue in an acceptable way. Most just dismiss it as trivial or take up pragmatic argumentation which is in itself unlibertarian. Murray Rothbard devotes a chapter of his Ethics of Liberty to it, but even there I do not find all the answers I seek. According to my conclusions children should be treated in the exact same way adults are, otherwise some libertarian axiom is contradicted. If we restrict children's freedom we immediately equate them with slaves. That would set a terrible precedent for other situations where it would suddenly become acceptable to enslave people for pragmatic reasons.
I find myself, as on most other issues, siding exactly with the position of the great Herbert Spencer. As he wrote in Social Statics (the most important book ever written), "Those commonly-observed facts, that the enslavement of woman is invariably associated with a low type of social life, and that conversely, her elevation towards an equality with man uniformly accompanies progress, were cited in part proof that the subjection of female to male is essentially wrong. If now, instead of women we read children, similar facts may be cited, and a similar deduction may be drawn."
Spencer is correct in that progress in society can only be measured by the level of coercion exercised within it - the less coercion, the more the progress. Thus it is right to treat children as if they were just a human being, without applying special categories to them. After all, age is just a category. Similar ones have been applied to the sexes or races in the past. However, I don't think I need to mention the repercussions of such an attitude turning into policy. There is clearly something wrong with children being able to act as adults. In such a case a child could sign a contract which would make him a slave while he is 3 years old and doesn't know how to read yet! The only other option I see is treating children as incapacitated citizens (as if they were mentally ill or something such) until they reach a certain age or level of ability. These standards would have to be set by individuals or communities on a local basis.
I am always willing to listen to proposals on this issue!