Look at the wires again and see if there's any writing on either of the wires. The one that has writing on it is also the neutral wire. Otherwise, as lazypup mentioned, get yourself a multimeter and check for continuity to make sure which wire is which.
This diagram will show you the importance of having the wires connected correctly and how to check that they are.
As for your junction box, you should be able to see a screw or nail going through it that attaches it to a joist. If it's a metal box then you may be able to install another screw into the box and through the joist to secure it better. Or, if you have access to the attic then it would be wise to go up there and try to secure it better.