Working with a team ( not directly reporting into pm): challenges for project managers?

Yes ... this could be a big challenge.  However, it is now common to see resources coming on temporary basis to work on a project as project team members for a specific task and not for the whole duration of the project.

Project managers may outsource parts of the project to external entities outside their direct control.  However, reporting on project work/activities/progress/concerns/issues related to the project should go to the project manager ... as well as ...to the direct functional manager these resources belong to and report to !