This repository has been archived on 2023-11-29. You can view files and clone it, but cannot push or open issues or pull requests.
WebPicDownloader/README.md

108 lines
5.4 KiB
Markdown
Raw Permalink Normal View History

2022-09-01 20:51:00 +02:00
# WebPicDownloader
2022-09-12 22:33:56 +02:00
[![Donate][link-icon-coffee]][link-paypal-me] [![Website][link-icon-website]][link-website]
[link-icon-coffee]: https://img.shields.io/badge/%E2%98%95-Buy%20me%20a%20cup%20of%20coffee-991481.svg
[link-paypal-me]: https://www.paypal.me/EndMove/2.5eur
[link-icon-website]: https://img.shields.io/badge/%F0%9F%92%BB-My%20Web%20Site-0078D4.svg
[link-website]: https://www.endmove.eu/
2022-09-06 21:41:03 +02:00
## What is webpicdownloader ?
WebPicDownloader is a scraping tool that allows you to download all the images of a website. Basically WebPic is a Python script around which a graphical interface has been added to make it easier to use.
2022-09-12 22:33:56 +02:00
You will find [here](#windows-application) utility information to use the Windows application `WebPicDownloader.exe`. And [here](#use-python-script) information to use or implement the Python script `WebPicDownloader.py` in your application (without the graphical interface).
2022-09-06 21:41:03 +02:00
## Windows application
2022-09-12 22:33:56 +02:00
To use WebPic on windows nothing more simple, download the executable `.exe` of the last [release here](https://git.endmove.eu/EndMove/WebPicDownloader/releases) (be careful to download the latest release and not a pre-release).
2022-09-06 21:41:03 +02:00
Execute the file ``WebPicDownloader.exe`` and enjoy it! 👌
## Use Python script
2022-09-12 22:33:56 +02:00
To start, find the script to use or to add to your code [here](webpicdownloader/model/WebPicDownloader.py).
2022-09-06 21:41:03 +02:00
2022-09-12 22:33:56 +02:00
### CLI Run Requirements
2022-09-06 21:41:03 +02:00
To use the script check the following prerequisites.
* Python `>= 3.10.6` ;
2022-09-11 11:52:41 +02:00
* beautifulsoup4 `>= 4.11.1` ;
2022-09-06 21:41:03 +02:00
* bs4 (BeautifulSoup) `>= 0.0.1` ;
2022-09-11 11:52:41 +02:00
* urllib3 `>= 1.26.12` ;
2022-09-06 21:41:03 +02:00
### Console Use ?
2022-09-12 22:33:56 +02:00
If you just want to use the console version of the script without the built-in GUI then you just need to check the [prerequisites](#cli-run-requirements) and run the script as follows:
2022-09-06 21:41:03 +02:00
```python
python3 WebPicDownloader.py
```
2022-09-12 22:33:56 +02:00
### Integrate to your code ?
First of all you have to know that WebPicDownloader has a deamon worker that downloads all the images asynchronously (this allows you not to block your program when a download is in progress). This same worker will be automatically killed as soon as your program finishes. WebPicDownloader therefore provides a blocking stop function allowing you to wait for the end of the download. See the information below. The prerequisites are the same as if you were running the script from the command line, see [prerequisites](#cli-run-requirements).
2022-09-12 22:41:11 +02:00
#### Step 1
2022-09-12 22:33:56 +02:00
Instantiate your WebPicDownloader object like this:
```python
from WebPicDownloader import WebPicDownlodaer, MessageType
webpic = WebPicDownloader()
```
The constructor can take several parameters (`path: str, headers: dict, messenger, success, failure`) (see the documentation).
2022-09-12 22:41:11 +02:00
#### Step 2
2022-09-12 22:33:56 +02:00
Define the WebPicDownloader callback functions. There are 3 main ones, the first (messenger callback) will be called at each system event and takes the following parameters (`message: str, type: MessageType`). The second (success callback) will be called at the end of processing if no major errors occur, it takes the following parameters (`message: str`). The third and last function (failure callback) will be called if a major error occurs or the download fails, it takes the following parameter (`message: str`).
By default, these functions print their results with a simple `print(message)` in the console. In case you implement WebPicDownloader in a graphical program, you should by convention remove all printing from your application and therefore define your own callback functions for WebpicDownloder. Below is an example:
```python
from WebPicDownloader import WebPicDownlodaer, MessageType
# Consider instantiating before the main loop of your program is launched.
webpic = WebPicDownloader()
# Pay attention to the signature of the functions
webpic.set_success_callback(lambda message: print(f"Success ! [{message}]."))
webpic.set_failure_callback(lambda message: print(f"Success ! [{message}]."))
webpic.set_messenger_callback(lambda message, msg_type: print(f"[{msg_type}]: {message}."))
```
2022-09-12 22:41:11 +02:00
#### Step 3
2022-09-12 22:33:56 +02:00
Once WebPicDownloader instantiated and the callback functions configured, we have to launch the download and stop it. It is important to know that the script does not have a function to stop a download in progress, in fact the stop function will allow you to wait for the end of the download and then turn off the program or to kill the worker automatically when the main thread dies.
```python
from time import sleep
from WebPicDownloader import WebPicDownlodaer, MessageType
webpic = WebPicDownloader()
# ... callbacks ...
# Webpic will give the task to its worker and start downloading the images
webpic.start_downloading('https://www.endmove.eu/', 'EndMove-website-images')
# We wait for the worker to start the task (once the task has started it cannot be stopped)
sleep(1)
# Webpic will ask the program to stop in blocking mode (it will join the worker to wait for the end of its execution)
webpic.stop_downloading(True)
```
## Improvement (TODO LIST)
Here you will find some improvements I would like to add to the program, you can also participate by forking the repository and submitting a pull request.
- [x] Check for updates button.
- [ ] Integrated file explorer.
- [ ] Viewing the downloads already made.
- [ ] Redo WebPicDownlodaer script to support concurrent downloads, to be able to launch workers and share tasks via a download pool.
2022-09-06 21:41:03 +02:00
2022-09-12 22:33:56 +02:00
This program is only a free utility tool and has not been developed in depth. In a future version it would be interesting to manage concurrent downloads in a thread pool.