Skip to content

Tag: lxml

Python and LXML: extremely slow, more efficient code?

I’m processing XML documents like the following. I’m using the following code to change the attributes of certain elements whenever certain conditions are met. The code works as expected and I’m getting the output I want. However the time it takes to process all the files seems way too much.…

Get NA for empty slots in lxml xpath()

I have a big xml (that one): of which I am providing a sample here: I now want to pull out all biospecimen and concentration_value and be able to associate them with each other in the end. I am trying to do it like this: The output csv should look like this: In reality I also pull out many other

Parsing a XML child element back as a string

I’m trying to parse a complex XML and xpath isn’t behaving like I thought it would. Here’s my sample xml: Here’s my python code: I get the output: when I expected: What am I doing wrong? Answer This XPath will get text and elements as expected Printing found nodes as OP requested Resul…

Problems installing lxml on M1 mac

So, I’m having the classic trouble install lxml. Initially I was just pip installing, but when I tried to free up memory using Element.clear() I was getting the following error: I thought this must be because lxml is using the system’s libxml2 which is probably out of date. So I used homebrew to i…

Scrapy Python can‘t extract links with more stable xpath

I‘m Building a scraper for this website. I‘m using Python and scrapy Shell to extract the data that I want: xpath would be: //a[@class=“sb-card sb-card-company site-1×1 with-hover]/@href“ Using response.xpath(‘//a[@class=“sb-card sb-card-company site-1×1 with-hover]/@href“‘ returns [] I tried using …

Python lxml find text efficiently

Using python lxml I want to test if a XML document contains EXPERIMENT_TYPE, and if it exists, extract the <VALUE>. Example: Is there a faster way than iterating through all elements? That attempt is also getting messy when I want to extract the <VALUE>. Answer Preferably you do this with XPath wh…

I want to replace the html code with my own

I am using lxml and beautifulsoup library, actually my goal is to translate text of the specific tags out of the whole html code, what I want is, I want to replace the text of specific tags with the translated text. I want to set a loop for the specific xpath in which all the translated text should be inserte…