Do they know whos the boss in the office? I have had this experience myself that the boss is working in another room alone, and many people use their working time watching youtube and myspace etc etc. From what i have heard, and i guess it can be done, is that the IT department can create special log in accounts, where you have access only on certain sites. But that wont work for you at this time, since your IT guy is gone.
But tell me, are you in the same room with them? I am pretty certain if you would be, they would be more productive. But never the less, i find your situation quite retarded, especially if those people have kids at home. They should be happy they actually have work to do and get paid.
Giving out a serious warning or making a meeting like someone said before, is not very good idea, because it will cause tension, and you could be in a "they" against "you" situation.
Never the less, if you say your clients have problems, do your workers know about the deadlines? To me it sounds if they dont know really.