Richard Henry Lee who wrote as the Federal Farmer said "to lay and collect internal taxes, in this extensive country, must require a great number of congressional ordinances, immediately operating upon the body of the peoples; these must continually interfere with the state laws, and thereby produce disorder and general dissatisfaction, till the one system of laws or the other, operating upon the same subjects, shall be abolished...Further, as to internal taxes, the state governments will have concurrent powers with the general government, and both may tax the same objects in the same year;and the objection that the general government may suspend a state tax, as a necessary measure for the promoting the collection of a federal tax, is not without foundation."
Brutus in Anti-Federalists No. 5 writes: "To detail the particulars comprehended in the general terms, taxes, duties, imposts and excises, would require a volume, instead of a single piece in a news-paper. Indeed it would be a task far beyond my ability, and to which no one can be competent, unless possessed of a mind capable of comprehending every possible source of revenue; for they extend to every possible way of raising money, whether by direct or indirect taxation. Under this clause may be imposed a poll-tax, a land-tax, a tax on houses and buildings, on windows and fire places, on cattle and on all kinds of personal property: — It extends to duties on all kinds of goods to any amount, to tonnage and poundage on vessels, to duties on written instruments, newspapers, almanacks, and books: — It comprehends an excise on all kinds of liquors, spirits, wines, cyder, beer, etc. and indeed takes in duty or excise on every necessary or conveniency of life; whether of foreign or home growth or manufactory. In short, we can have no conception of any way in which a government can raise money from the people, but what is included in one or other of three general terms. We may say then that this clause commits to the hands of the general legislature every conceivable source of revenue within the United States. Not only are these terms very comprehensive, and extend to a vast number of objects, but the power to lay and collect has great latitude; it will lead to the passing a vast number of laws, which may affect the personal rights of the citizens of the states, expose their property to fines and confiscation, and put their lives in jeopardy: it opens a door to the appointment of a swarm of revenue and excise officers to pray [sic] upon the honest and industrious part of the community, eat up their substance, and riot on the spoils of the country. We will next enquire into what is implied in the authority to pass all laws which shall be necessary and proper to carry this power into execution. It is, perhaps, utterly impossible fully to define this power. The authority granted in the first clause can only be understood in its full extent, by descending to all the particular cases in which a revenue can be raised; the number and variety of these cases are so endless, and as it were infinite, that no man living has, as yet, been able to reckon them up. The greatest geniuses in the world have been for ages employed in the research, and when mankind had supposed that the subject was exhausted they have been astonished with the refined improvements that have been made in modem times, and especially in the English nation on the subject — If then the objects of this power cannot be comprehended, how is it possible to understand the extent of that power which can pass all laws which shall be necessary and proper for carrying it into execution? It is truly incomprehensible. A case cannot be conceived of, which is not included in this power. It is well known that the subject of revenue is the most difficult and extensive in the science of government. It requires the greatest talents of a statesman, and the most numerous and exact provisions of the legislature. The command of the revenues of a state gives the command of every thing in it. — He that has the purse will have the sword, and they that have both, have every thing; so that the legislature having every source from which money can be drawn under their direction, with a right to make all laws necessary and proper for drawing forth all the resource of the country, would have, in fact, all power. Were I to enter into the detail, it would be easy to shew how this power in its operation, would totally destroy all the powers of the individual states. But this is not necessary for those who will think for themselves, and it will be useless to such as take things upon trust, nothing will awaken them to reflection, until the iron hand of oppression compel them to it."
William Symmes Jr. stated that the term general welfare might be applied to any expenditure whatever instead of just the enumerated powers of congress.
Thomas Jefferson wrote in a letter in June of 1817: "You will have learned that an act for internal improvement, after passing both houses, was negatived by the President. The act was founded, avowedly, on the principle that the phrase in the constitution, which authorizes Congress 'to lay taxes, to pay the debts and provide for the general welfare,' was an extension of the powers specifically enumerated to whatever would promote the general welfare; and this, you know, was the federal doctrine. Whereas, our tenet ever was, and, indeed, it is almost the only land-mark which now divides the federalists* from the republicans, that Congress had not unlimited powers to provide for the general welfare, but were restrained to those specifically enumerated; and that, as it was never meant they should provide for that welfare but by the exercise of the enumerated powers, so it could not have been meant they should raise money for purposes which the enumeration did not place under their action: consequently, that the specification of powers is a limitation of the purposes for which they may raise money."
Patrick Henry who first started demanding taxation be done only at the local level in 1765 said this in the Stamp Act: "Resolved, therefore, That the General Assembly of this Colony, together with his Majesty or his Substitutes, have, in their Representative Capacity, the only exclusive Right and Power to lay Taxes and Imposts upon the Inhabitants of this Colony: And that every Attempt to vest such Power in any other Person or Persons whatever, than the General Assembly aforesaid, is illegal, unconstitutional and unjust, and have a manifest Tendency to destroy British as well as American Liberty."
It is clear while reading through these quotes from these Founding Fathers that they were worried about this clause being to vague and that at some future point in America that the general government would abuse it for their own purposes instead of defense of the States. Some even went so far as to try to have this clause removed from the Constitution before ratification. Time has only shown these Patriots to be correct in that congress would trample States' Rights through taxation and other means by declaring that this clause makes it constitutional.