Browse Source

Fixed #23

Abrax 3 years ago
1 changed files with 10 additions and 83 deletions
  1. 10

+ 10
- 83 View File

@@ -16,59 +16,27 @@ This version will change syntax for some commands:
These changes take effect with version 0.4 so be ready to update your scripts!

### 0.3 (current)
This version changes syntax for some commands:

* Database creation: `lizard c` becomes `lizard create`
* Conservative refresh:
* `lizard rc` no longer a valid command
* `lizard r` does conservative refresh
* Refresh all threads: `lizard ra` instead of `lizard r`
* This version changes syntax for some commands:
* Database creation: `lizard c` becomes `lizard create`
* Conservative refresh:
* `lizard rc` no longer a valid command
* `lizard r` does conservative refresh
* Refresh all threads: `lizard ra` instead of `lizard r`
* Changed packaging system: Now building wheels which are compatible with `pip`
* Basic Tor support:
* Basic Tor support:
* `lizard to` will open the threads using the onion domain.
* `lizard tor` will also open the CAPTCHA page

### 0.2
* Help message now shows version. (issue #24)
* New command: `rl` is refresh + list
* New command: `ro` is refresh + open in browser
* `c` and `b` now print more useful output.
* Better handling of non-existent/invalid database.

### 0.1.1
* Now explicitly specifying python3 in script header, so it shouldn't attempt running with python2.
* `lizard e` output format matches the new packaged system.
### Older versions

### 0.1
- Introduced package system.
See [wiki](

## Installation
### Linux

1. Download the most recent package from under `dist/`

2. Install with pip: `pip install lizard-0.<VERSION HERE>-py3-none-any.whl`

This will also upgrade to a newer version if you have an older version installed.

To uninstall, you can do `pip uninstall lizard`.

### Upgrading
If you've been using Lizard for a while and are upgrading to a new release, be very careful with your local data. When a new version changes the database code or how local files are stored, it could fail to read your old shit and even corrupt it. I try to note this somewhere when it happens, but I may forget or you may not see the announcement. If in doubt, backup everything (all of Lizard's data is stored under `~/lizard_data/`) before trying the new version.

If you know for sure that a version breaks backwards compatibility, you have the following options:

* Just say fuck it and start over with fresh database.
* Run `lizard e` in your old lizard, save the resulting file, install the new version and then re-construct your watch list by running the exported commands. If you had any unseen replies or something you might lose those, so I'd check first with `lizard r` and `lizard l`. It might also re-download the old files if I changed where the files are stored. If you can figure out where they are *supposed* to go you can copy them over to avoid the re-downloading.
* Open the databases in another program (see FAQ) and manually migrate the data.
* Beg me to make a migration script.

### Running sources directly
You can also run the `.py` files in the repo directly. If you want to do this I'm assuming you have enough proficiency with Python to figure it out. FYI, I use PyCharm to develop so it might be easier to get it to work with it.

#### Building

`` is convenient for creating a package.
More detailed install info at [wiki](If upgrading, see [wiki](

## Usage
Running `lizard` without any arguments will print the help message explaining the syntax. It will also create a `~/lizard_data` directory to store files.
@@ -76,44 +44,3 @@ Running `lizard` without any arguments will print the help message explaining th
After you install, run `lizard create` to create a new database (if you don't have one already). Everything else will crash unless a valid database exists.

Add a thread to the database with `lizard a <URL>`. Upon adding a thread, Lizard will immediately download a copy of the thread and the files in it. These will be put under `~/lizard_data/`.

## Troubleshooting
Make sure you have Python 3.6 and are using it to run `lizard`. Python 2 will absolutely not run it, and the libraries I use may cause problems with older versions of Python 3.

I generate the release archive with Python's `distutils`. I have no idea how it deals with dependencies on other Python packages. You can install these with `pip install <package name>`. Off the top of my head you need `peewee` and `humanize`. I also use `os`, `shutil`, `webbrowser`, `json`, `requests` and `re` but these should be included in Python3 by default.

### Does Lizard work at all?
Simply running `lizard` without any commands will print a help message (and also ensure that `~/lizard_data/` exists). If you can't get this to happen, the problem is probably your environment.

Next, try to run `lizard create`. It should create a new database, but if one already exists you need to get rid of it (either delete or rename). `lizard b` will back up the DB for you, but it won't erase the current one. Make sure the database it creates is `~/lizard_data/threads.orm.db` and not somewhere else.

If `lizard create` is able to create a database, then see if you can add a thread. Try `lizard a <URL of a thread>` for a few different threads. If none of them worked, again, probably something is fucked in your Lizard installation. If all worked, great! Lizard should be working correctly. If only some threads work, it's probably because of a bug specific to those I didn't know about, submit an issue and include steps to reproduce.

### Corrupted database
If your existing `threads.orm.db` file causes problems, but a fresh one works fine, either your database is just fucked or one of the threads in it is causing problems. You can elminate the latter possibility by re-adding the threads to the fresh database. If there's a thread that consistently causes problems, congratulations, you found a bug! Submit an issue and wait for me to fix it, meanwhile don't add that thread. If you manage to re-add all the threads to the fresh database and everything works, just delete the old database and forget about it. Life's too short, man.

## FAQ
### Is `lizard` only for Linux?
Since Python is cross platform it should run if Python runs, but I only use it on Linux so currently instructions are for that. They shouldn't be hard to adapt to other OSes, but if anyone is actually interested in running this on something else I can try to provide specific instructions.

### Is there a GUI?
No. An earlier version of this program had a GUI, but it turned out to be more trouble than it's worth.

### Why doesn't Lizard automatically refresh threads at set intervals?
The user is expected to use the scheduling system of their OS to accomplish this.

For instance, on Linux you can set up `cron` to run `lizard rc` regularly, and maybe redirect output to your own log file as well.

### How can I inspect the database?
`lizard l` will print out a summary, but if you want the nitty gritty you need a SQLite browser such as [sqliteman]( or [DB Browser for SQLite](

### Why does it create so many `.json` and `.html` files in the data directory?
These are saved at each refresh in case the thread dies or any replies are deleted. Deleting these files when they are no longer needed is the user's responsibility.

### How do you handle file names?
I use the original filenames with the first 10 characters of 8chan's hash-based name. Adding the hash is necessary because 8ch allows duplicate filenames in the same thread, and they would otherwise conflict with each other when saved with the original name.

### How does the conservative refresh predict replies?
If the last reply was `x` seconds ago, and the thread was last refreshed `y` seconds ago, `lizard` doesn't bother refreshing unless `x/y>10`. Internally, the 10 is called `conservative_refresh_criterion`. Increasing this will make it less conservative, decreasing will make it more conservative.

In reality I think replies would follow a bimodal (people who browse only the front page and people who browse the catalog) or trimodal (bimodal plus people who watch individual threads, such as with this tool) distribution, with each mode itself being a Poisson. Interesting as that is, I think my simple heuristic works well enough so I'm not planning to implement such rigorous statistical analysis for the time being.