Skip to content
Advertisement

How do you validate a URL with a regular expression in Python?

I’m building an app on Google App Engine. I’m incredibly new to Python and have been beating my head against the following problem for the past 3 days.

I have a class to represent an RSS Feed and in this class I have a method called setUrl. Input to this method is a URL.

I’m trying to use the re python module to validate off of the RFC 3986 Reg-ex (http://www.ietf.org/rfc/rfc3986.txt)

Below is a snipped which should work? 

p = re.compile('^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(?([^#]*))?(#(.*))?')
m = p.match(url)
if m:
  self.url = url
  return url

Advertisement

Answer

An easy way to parse (and validate) URL’s is the urlparse (py2, py3) module.

A regex is too much work.

There’s no “validate” method because almost anything is a valid URL. There are some punctuation rules for splitting it up. Absent any punctuation, you still have a valid URL.

Check the RFC carefully and see if you can construct an “invalid” URL. The rules are very flexible.

For example ::::: is a valid URL. The path is ":::::". A pretty stupid filename, but a valid filename.

Also, ///// is a valid URL. The netloc (“hostname”) is "". The path is "///". Again, stupid. Also valid. This URL normalizes to "///" which is the equivalent.

Something like "bad://///worse/////" is perfectly valid. Dumb but valid.

Bottom Line. Parse it, and look at the pieces to see if they’re displeasing in some way.

Do you want the scheme to always be “http”? Do you want the netloc to always be “www.somename.somedomain”? Do you want the path to look unix-like? Or windows-like? Do you want to remove the query string? Or preserve it?

These are not RFC-specified validations. These are validations unique to your application.

Advertisement