In order to better understand how native English speakers speak, you must Understand that written, classroom English is different than natural, spoken English. When you are practicing your speaking and listening skills, recognize that you are learning a different language. Learn the rules of English pronunciation–especially stress, rhythm and connected speech. Practice transforming the language with these rules yourself. Listen to/watch videos of native-level English speakers while reading the transcript (I recommend TED.com for this).

Re-listen to and take notice of how the speakers are transforming their language, and how this compares to careful written English. The more you understand the differences between written English and spoken, the more you can train your ears to listen to native speakers and understand them.