python urllib.urlopen

This is my first python program.
This works with python2 language but not work with python3 so to work together install:
https://www.python.org/ftp/python/2.7.5/python-2.7.5.msi
0006d6219160ce6abe711a71c835ebb0
52936e98404349d02e64fec710d209f99197ce86

This is first example. This will show html source similar to wget -qO- https://www.google.com/
Use your favorite text editor to save this as first.py:
import urllib
htmlfile = urllib.urlopen("https://www.google.com/")
htmltext = htmlfile.read()
print htmltext
Open "IDLE (Python GUI)"
Choose File -> Open:
Actually now you can open every python (py) file with python editor:
When the python code is loaded you can run the program by pressing F5 or choose Run -> Run Module:
Second example will create an url array and print out only first 100 characters of each html filesecond.py
import urllib
urls = ["https://www.google.com/", "https://edition.cnn.com/"]
i=0
while i< len(urls):
	htmlfile = urllib.urlopen(urls[i])
	htmltext = htmlfile.read()
	print htmltext[1:100]
	i+=1
Third example will extract page titles. Keep the tabs in while cycle otherwise nothing will work out.
third.py
import urllib
import re
urls = [https://www.google.com/", "https://edition.cnn.com/"]
i=0
regex = '<title>(.+?)</title>'
pattern = re.compile(regex)
while i< len(urls):
	htmlfile = urllib.urlopen(urls[i])
	htmltext = htmlfile.read()
	titles = re.findall(pattern,htmltext)
	print titles
	i+=1

No comments: