本文實(shí)例講述了Python基于lxml模塊解析html獲取頁(yè)面內(nèi)所有葉子節(jié)點(diǎn)xpath路徑功能。分享給大家供大家參考，具體如下：

因?yàn)樾枰褂萌~子節(jié)點(diǎn)的路徑來(lái)作為特征，但是原始的lxml模塊解析之后得到的卻是整個(gè)頁(yè)面中所有節(jié)點(diǎn)的xpath路徑，不是我們真正想要的形式，所以就要進(jìn)行相關(guān)的處理才行了，差了很多網(wǎng)上的博客和文檔也沒(méi)有找到一個(gè)是關(guān)于輸出html中全部葉子節(jié)點(diǎn)的API接口或者函數(shù)，也可能是自己沒(méi)有那份耐心，沒(méi)有找到合適的資源，只好放棄了尋找，但是這并不說(shuō)明沒(méi)有其他的方法了，在對(duì)頁(yè)面全部節(jié)點(diǎn)的xpath輸出之后觀察得到的結(jié)果就是：

1.路徑之間存在包含性

2.葉子節(jié)點(diǎn)的路徑必然包含上一個(gè)葉子節(jié)點(diǎn)路徑到下一個(gè)葉子節(jié)點(diǎn)路徑之間的路徑

3.所有的葉子節(jié)點(diǎn)均不存在包含性

基于這些觀察就可以實(shí)踐了，我采用的方法是，設(shè)置一個(gè)標(biāo)志位，如果當(dāng)前路徑被下一條路徑包含的話就把下標(biāo)加1處理，直到遇上不包含的情況是這一條路徑就是一條葉子節(jié)點(diǎn)路徑，加入結(jié)果列表即可，經(jīng)檢驗(yàn)和原始路徑列表對(duì)比后，結(jié)果路徑列表均不存在包含性，暫可認(rèn)為均為葉子節(jié)點(diǎn)的路徑，不敢確定的說(shuō)是因?yàn)椋壕箾](méi)有大量做實(shí)驗(yàn)觀察是否有另類，不過(guò)方法是可行的，下面是具體實(shí)現(xiàn)：

#!usr/bin/env python
#encoding:utf-8
'''''
__author__:沂水寒城
功能：得到頁(yè)面的葉子節(jié)點(diǎn)的xpath
'''
def get_leaf_node_xpath(one_page_xpath):
  '''''
  輸入：一個(gè)頁(yè)面的原始xpath路徑列表
  輸出：只包含頁(yè)面的葉子節(jié)點(diǎn)的xpath列表
  '''
  one_page_xpath.append('0')
  leaf_node_xpath_list=[]
  for i in range(len(one_page_xpath)-1):
    j=i+1
    one_xpath=one_page_xpath[i]
    two_xpath=one_page_xpath[j]
    if one_xpath in two_xpath:
      one_xpath=two_xpath
      two_xpath=one_page_xpath[j+1]
    else:
      leaf_node_xpath_list.append(one_xpath)
  return leaf_node_xpath_list
if __name__ == '__main__':
  with open('baidu.txt') as f:
    html=f.read()
  htree, one_page_xpath=get_clean_allnodes_xpath(html)
  leaf_node_xpath_list=get_leaf_node_xpath(one_page_xpath)
  print leaf_node_xpath_list
  with open('data/dom_leaf_xpath/original_xpath.txt', 'w') as f1:
    for one_line in one_page_xpath:
      f1.write(one_line.strip()+'\n')
  with open('data/dom_leaf_xpath/leaf_xpath.txt', 'w') as f2:
    for one_line in leaf_node_xpath_list:
      f2.write(one_line.strip()+'\n')

打開保存的文件內(nèi)容為：

original_xpath.txt內(nèi)容為：

/html
/html/head
/html/head/meta[1]
/html/head/meta[2]
/html/head/meta[3]
/html/head/meta[4]
/html/head/title
/html/body
/html/body/p
/html/body/p/comment()[1]
/html/body/p/comment()[2]
/html/body/p/comment()[3]
/html/body/p/meta
/html/body/div[1]
/html/body/div[1]/div[1]
/html/body/div[1]/div[1]/div
/html/body/div[1]/div[1]/div/div[1]
/html/body/div[1]/div[1]/div/div[1]/div
/html/body/div[1]/div[1]/div/div[1]/div/div[1]
/html/body/div[1]/div[1]/div/div[1]/div/a
/html/body/div[1]/div[1]/div/div[1]/div/form
/html/body/div[1]/div[1]/div/div[1]/div/form/span[1]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[2]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/div
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/div/span
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[1]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[1]/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[2]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[2]/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[3]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[4]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[4]/a
/html/body/div[1]/div[1]/div/div[1]/div/div[2]
/html/body/div[1]/div[1]/div/div[2]
/html/body/div[1]/div[1]/div/div[2]/a[1]
/html/body/div[1]/div[1]/div/div[2]/a[2]
/html/body/div[1]/div[1]/div/div[2]/a[3]
/html/body/div[1]/div[1]/div/div[3]
/html/body/div[1]/div[1]/div/div[3]/a[1]
/html/body/div[1]/div[1]/div/div[3]/a[2]
/html/body/div[1]/div[1]/div/div[3]/a[3]
/html/body/div[1]/div[1]/div/div[3]/a[4]
/html/body/div[1]/div[1]/div/div[3]/a[5]
/html/body/div[1]/div[1]/div/div[3]/a[6]
/html/body/div[1]/div[1]/div/div[3]/a[7]
/html/body/div[1]/div[1]/div/div[3]/a[8]
/html/body/div[1]/div[1]/div/div[3]/a[9]
/html/body/div[1]/div[2]
/html/body/div[1]/div[2]/a[1]
/html/body/div[1]/div[2]/a[2]
/html/body/div[1]/div[2]/a[3]
/html/body/div[1]/div[2]/a[4]
/html/body/div[1]/div[2]/a[5]
/html/body/div[1]/div[2]/a[6]
/html/body/div[1]/div[2]/a[7]
/html/body/div[1]/div[2]/a[8]
/html/body/div[1]/div[2]/a[9]
/html/body/div[1]/div[3]
/html/body/div[1]/div[3]/div
/html/body/div[1]/div[3]/div/div
/html/body/div[1]/div[3]/div/div/div[1]
/html/body/div[1]/div[3]/div/div/div[2]
/html/body/div[1]/div[3]/div/div/div[2]/p
/html/body/div[1]/div[4]
/html/body/div[1]/div[4]/div
/html/body/div[1]/div[4]/div/div
/html/body/div[1]/div[4]/div/div/p[1]
/html/body/div[1]/div[4]/div/div/p[1]/a[1]
/html/body/div[1]/div[4]/div/div/p[1]/a[2]
/html/body/div[1]/div[4]/div/div/p[1]/a[3]
/html/body/div[1]/div[4]/div/div/p[1]/a[4]
/html/body/div[1]/div[4]/div/div/p[2]
/html/body/div[1]/div[4]/div/div/p[2]/a[1]
/html/body/div[1]/div[4]/div/div/p[2]/a[2]
/html/body/div[1]/div[4]/div/div/p[2]/a[3]
/html/body/div[1]/div[5]
/html/body/div[2]
/html/body/div[3]
/html/body/div[4]
0

最后一行的0只是為了代碼中列表能夠遍歷到最后一個(gè)路徑而不出現(xiàn)索引錯(cuò)誤的一個(gè)簡(jiǎn)單的處理，人為的添加了一個(gè)元素，沒(méi)有實(shí)際意義，也不會(huì)參與處理

leaf_xpath.txt內(nèi)容為：

/html/head/meta[1]
/html/head/meta[2]
/html/head/meta[3]
/html/head/meta[4]
/html/head/title
/html/body/p/comment()[1]
/html/body/p/comment()[2]
/html/body/p/comment()[3]
/html/body/p/meta
/html/body/div[1]/div[1]/div/div[1]/div/div[1]
/html/body/div[1]/div[1]/div/div[1]/div/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[1]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[2]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/div/span
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[1]/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[2]/a
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[3]
/html/body/div[1]/div[1]/div/div[1]/div/form/span[3]/span/ul/li[4]/a
/html/body/div[1]/div[1]/div/div[1]/div/div[2]
/html/body/div[1]/div[1]/div/div[2]/a[1]
/html/body/div[1]/div[1]/div/div[2]/a[2]
/html/body/div[1]/div[1]/div/div[2]/a[3]
/html/body/div[1]/div[1]/div/div[3]/a[1]
/html/body/div[1]/div[1]/div/div[3]/a[2]
/html/body/div[1]/div[1]/div/div[3]/a[3]
/html/body/div[1]/div[1]/div/div[3]/a[4]
/html/body/div[1]/div[1]/div/div[3]/a[5]
/html/body/div[1]/div[1]/div/div[3]/a[6]
/html/body/div[1]/div[1]/div/div[3]/a[7]
/html/body/div[1]/div[1]/div/div[3]/a[8]
/html/body/div[1]/div[1]/div/div[3]/a[9]
/html/body/div[1]/div[2]/a[1]
/html/body/div[1]/div[2]/a[2]
/html/body/div[1]/div[2]/a[3]
/html/body/div[1]/div[2]/a[4]
/html/body/div[1]/div[2]/a[5]
/html/body/div[1]/div[2]/a[6]
/html/body/div[1]/div[2]/a[7]
/html/body/div[1]/div[2]/a[8]
/html/body/div[1]/div[2]/a[9]
/html/body/div[1]/div[3]/div/div/div[1]
/html/body/div[1]/div[3]/div/div/div[2]/p
/html/body/div[1]/div[4]/div/div/p[1]/a[1]
/html/body/div[1]/div[4]/div/div/p[1]/a[2]
/html/body/div[1]/div[4]/div/div/p[1]/a[3]
/html/body/div[1]/div[4]/div/div/p[1]/a[4]
/html/body/div[1]/div[4]/div/div/p[2]/a[1]
/html/body/div[1]/div[4]/div/div/p[2]/a[2]
/html/body/div[1]/div[4]/div/div/p[2]/a[3]
/html/body/div[1]/div[5]
/html/body/div[2]
/html/body/div[3]
/html/body/div[4]

針對(duì)這個(gè)問(wèn)題暫時(shí)可以想到的方法就是這樣的，利用包含性來(lái)解決，沒(méi)有其他的新思路，如果還有新的思路歡迎交流學(xué)習(xí)！

PS：這里再為大家提供幾款關(guān)于xml操作的在線工具供大家參考使用：

在線XML/JSON互相轉(zhuǎn)換工具：
http://tools.jb51.net/code/xmljson

在線格式化XML/在線壓縮XML：
http://tools.jb51.net/code/xmlformat

XML在線壓縮/格式化工具：
http://tools.jb51.net/code/xml_format_compress

XML代碼在線格式化美化工具：
http://tools.jb51.net/code/xmlcodeformat

更多關(guān)于Python相關(guān)內(nèi)容感興趣的讀者可查看本站專題：《Python操作xml數(shù)據(jù)技巧總結(jié)》、《Python數(shù)據(jù)結(jié)構(gòu)與算法教程》、《Python Socket編程技巧總結(jié)》、《Python函數(shù)使用技巧總結(jié)》、《Python字符串操作技巧匯總》、《Python入門與進(jìn)階經(jīng)典教程》及《Python文件與目錄操作技巧匯總》

希望本文所述對(duì)大家Python程序設(shè)計(jì)有所幫助。