You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1.6 KiB
1.6 KiB
Single file link data formatting (something)
It's split into several parts from out of which only 1 is fully controllable by you. Those parts are:
- Matching URL based on it's domain name and setting up the needed functions for upcoming tasks
- this function is your filehost link parser
- if it doesn't match with any of the hosts it will discard the link
- before it processes the URL it gets verified by urllib
- Fetching the contents of URL by calling a function url2bs(link) that will return a BeautifulSoup object
- Passes the object to your filehost parser function and assigns it to a private variable called result
- Task of your function is to return a 3-element array [state, url, headers]
- state - an integer that communicates the state of the link, it should contain values ranging from 0 to 2
- 0 = no data returned
- 1 = file link found
- 2 = folder link found
- if folder link offers a zip download function, please point it there appropriately and use state = 1 instead
- url - return a direct link to the file
- headers - headers necessary to install via _declare_opener(additional_header), this variable is optional and it's there for future-proofing
- Passes result to a file downloader function, it will pass a referrer (the referrer is the link itself) by default - If the result has state = 2, it will pass the result once more to a specialized function for parsing folders (your job as well) - The function will bypass the fifth step as well, you must call for the download functions yourself
- Downloads the file and job's done