using python to access web data week 5 assignment

Web Services and XML (Chapter 13)

eXtensible Markup Language

1. What is the name of the Python 2.x library to parse XML data?

  • xml-misc
  • xml.json
  • xml.etree.ElementTree
  • xml2

2. Which of the following are not commonly used serialization formats?

  • XML
  • JSON
  • Dictionaries
  • HTTP
  • TCP

3. In this XML, which are the "simple elements"?

<people>
<person>
<name>Chuck</name>
<phone>303 4456</phone>
</person>
<person>
<name>Noah</name>
<phone>622 7421</phone>
</person>
</people>
  • Noah
  • name
  • person
  • phone
  • people

4. In the following XML, which are attributes?

<person>
<name>Chuck</name>
<phonetype=”intl”>
+1 734 303 4456
</phone>
<emailhide=”yes”/>
</person>
  • hide
  • email
  • person
  • name
  • type

5. In the following XML, which node is the parent node of node e

<a>
<b>X</b>
<c>
<d>Y</d>
<e>Z</e>
</c>
</a>
  • c (Answer)
  • e
  • a
  • b

6. Looking at the following XML, what text value would we find at path "/a/c/e"

<a>
<b>X</b>
<c>
<d>Y</d>
<e>Z</e>
</c>
</a>
  • Z (Answer)
  • b
  • Y
  • a
  • e

7. What is the purpose of XML Schema?

  • To establish a contract as to what is valid XML
  • To transfer XML data reliably during network outages
  • To compute SHA1 checksums on data to make sure it is not modified in transit
  • A Python program to tranform XML files

8. If you were building an XML Schema and wanted to limit the values allowed in an xs:string field to only those in a particular list, what XML tag would you use in your XML Schema definition?

  • xs:complexType
  • xs:sequence
  • maxOccurs
  • xs:element
  • xs:enumeration

9. What does the "Z" mean in this representation of a time:

2002-05-30T09:30:10Z

  • This time is in the UTC timezone
  • The hours value is in the range 0-12
  • The local timezone for this time is New Zealand
  • This time is Daylight Savings Time

10. Which of the following dates is in ISO8601 format?

  • 2002-05-30T09:30:10Z
  • May 30, 2002
  • 05/30/2002
  • 2002-May-30

Extracting Data from XML

Extracting Data from XML In this assignment you will write a Python program somewhat similar to http://www.py4e.com/code3/geoxml.py. The program will prompt for a URL, read the XML data from that URL using urllib and then parse and extract the comment counts from the XML data, compute the sum of the numbers in the file. We provide two files for this assignment. One is a sample file where we give you the sum for your testing and the other is the actual data you need to process for the assignment. Sample data: http://py4e-data.dr-chuck.net/comments_42.xml (Sum=2553) Actual data: http://py4e-data.dr-chuck.net/comments_1913244.xml (Sum ends with 77) You do not need to save these files to your folder since your program will read the data directly from the URL. Note: Each student will have a distinct data url for the assignment - so only use your own data url for analysis. Data Format and Approach The data consists of a number of names and comment counts in XML as follows: Matthias 97 You are to look through all the tags and find the values sum the numbers. The closest sample code that shows how to parse XML is geoxml.py. But since the nesting of the elements in our data is different than the data we are parsing in that sample code you will have to make real changes to the code. To make the code a little simpler, you can use an XPath selector string to look through the entire tree of XML for any tag named 'count' with the following line of code: counts = tree.findall('.//count') Take a look at the Python ElementTree documentation and look for the supported XPath syntax for details. You could also work from the top of the XML down to the comments node and then loop through the child nodes of the comments node. Sample Execution $ python3 solution.py Enter location: http://py4e-data.dr-chuck.net/comments_42.xml Retrieving http://py4e-data.dr-chuck.net/comments_42.xml Retrieved 4189 characters Count: 50 Sum: 2... Turning in the Assignment Enter the sum from the actual data and your Python code below: Sum: (ends with 77) Python code:

import urllib.request
import xml.etree.ElementTree as ET

url = input(“Enter location: “)

uh = urllib.request.urlopen(url)
data = uh.read()
print(f”Retrieved {len(data)} characters”)

tree = ET.fromstring(data)
counts = tree.findall(‘.//count’)

total = sum([int(count.text) for count in counts])

print(“Count:”, len(counts))
print(“Sum:”, total)

Leave a Reply