The difference is, if you only have a permit, you cannot drive a CMV without a licensed driver sitting there, On Duty, right next to you. You can never be On Duty Driving while he's Off Duty in the sleeper berth, for example. He must remain On Duty whenever you are behind the wheel. So, he's On Duty while he's driving, and he's also On Duty while you're driving. That's essentially a solo driver, not a team operation.
Whether it's a good idea or not depends on whether you want to use the permit to obtain an actual CDL. If you do, then it's a great idea. It lets you learn how to drive a CMV in preparation for the CDL Road Test. A couple or three weeks (or months) driving with a permit should make the Road Test a breeze.
Also driving a CMV without a CDL, like, with just a permit and no licensed CDL holder sitting there next to you, could get you a civil penalty of up to US$2,500 per incidence or, in aggravated cases (traffic violation, accident), criminal penalties of up to US$5,000 in fines and/or up to 90 days in prison. A carrier is also subject to a penalty of up to US$10,000 if they knowingly permit a driver to operate a CMV without a valid CDL. That's why you'll be a Second Seat and not a Team Driver until you get your CDL.
In most states a CDL permit is good for 6 months, but in some they are good for a year. Far as I know, in the states where the permit is good for 6 months, you are allowed to take one road test on an expired permit, but the ones where the permit is good for a year you will need to take the road test before the permit expires, or else apply for another permit.
If someone is willing to take you on as a Second Seat with a permit, you should strongly considering doing so until you are ready to take the road test and obtain your CDL. Otherwise, there's always truck driving school.