add rate limit info print
update readme
This commit is contained in:
parent
ccb27e655b
commit
fc8daa01ec
48
README.md
48
README.md
@ -1,16 +1,22 @@
|
||||
# Zotero Addons Scraper
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
||||
This is a script repository for scraping Zotero addon collections for [Zotero](https://www.zotero.org), intended for use with [Zotero addons](https://github.com/syt2/zotero-addons).
|
||||
> Switch to `(addon-scraper)` source in [Zotero addons](https://github.com/syt2/zotero-addons) to use this repository.
|
||||
|
||||
The script utilizes the GitHub API and GitHub Actions to automatically scrape and parse addon information from the [addons](addons) folder, and publish to [`publish`](https://github.com/syt2/zotero-addons-scraper/blob/publish/addon_infos.json) branch and the latest release every day.
|
||||
This is a script repository for scraping [Zotero](https://www.zotero.org) add-ons, intended for use with [Zotero Addons](https://github.com/syt2/zotero-addons).
|
||||
> Switch to `addon-scraper` source in [Zotero Addons](https://github.com/syt2/zotero-addons) to use this repository.
|
||||
|
||||
## Contributing New Addons
|
||||
If you have a new add-on to add, add an add-on information file in the [addons](addons) folder.
|
||||
## Workflows
|
||||
This repository utilizes GitHub Actions workflow for automation.
|
||||
including the following steps:
|
||||
- Retrieve [Zotero](https://www.zotero.org) add-on information specified in [addons](addons) folder
|
||||
- Parse add-on information into JSON data
|
||||
- Publish the JSON data to the [`publish`](https://github.com/syt2/zotero-addons-scraper/blob/publish/addon_infos.json) branch and the [latest release](https://github.com/syt2/zotero-addons-scraper/releases/latest)
|
||||
|
||||
The file format should be as follows:
|
||||
## Contributing
|
||||
Adding a new add-on entry `{owner}#{repo}.json` in the [addons](addons) folder.
|
||||
e.g.,
|
||||
``` json
|
||||
{
|
||||
"name": "Zotero Addons",
|
||||
@ -22,27 +28,33 @@ The file format should be as follows:
|
||||
},
|
||||
{
|
||||
"targetZoteroVersion": "6",
|
||||
"tagName": "0.6.0-3"
|
||||
"tagName": "0.6.0-6"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `name`(optional):
|
||||
The name of the plugin.
|
||||
If not provided, the name will be extracted automatically from the XPI provided in release.
|
||||
Name of the add-on.
|
||||
If not provided, `name` will be extracted automatically from the XPI provided in release.
|
||||
- `repo`(required):
|
||||
The GitHub repository address of the plugin.
|
||||
GitHub repository of the plugin.
|
||||
- `releases`(required):
|
||||
Information about the plugin's XPI releases.
|
||||
Provide at least one valid plugin release information.
|
||||
XPI releases information of the Add-on.
|
||||
Provide at least one valid release information.
|
||||
- `targetZoteroVersion`(required):
|
||||
The Zotero compatibility version for the plugin.
|
||||
Zotero compatibility version for the add-on.
|
||||
Supports `"6"` or `"7"`.
|
||||
- `tagName`(required):
|
||||
The release information in the GitHub repository.
|
||||
Supports `"latest"`, `"pre"` or a specified tag name.
|
||||
If `"latest"` or `"pre"` is used, the script will automatically retrieve the latest release information.
|
||||
The release tag name in the GitHub repository.
|
||||
Supports `"latest"`, `"pre"` or `a specified tag name`.
|
||||
*If `"latest"` or `"pre"` is used, script will automatically retrieve the latest release tag.*
|
||||
|
||||
|
||||
## Usage
|
||||
1. `fork` this repository
|
||||
2. Enable actions and scheduled workflow in `Actions` page of forked repository
|
||||
3. Add a `Deploy key` in forked repository and named it with `ACTIONS_DEPLOY_KEY`
|
||||
|
||||
## Generated Format
|
||||
The output format for add-on information follows the format specified in [zotero-chinese/zotero-plugins](https://github.com/zotero-chinese/zotero-plugins).
|
||||
|
||||
12
main.py
12
main.py
@ -317,6 +317,15 @@ def delete_cache(github_repository, github_token, remain_count=2):
|
||||
print(f'get caches failed: {e}')
|
||||
|
||||
|
||||
def rate_limit(github_token):
|
||||
try:
|
||||
resp = requests.get('https://api.github.com/rate_limit', headers=github_api_headers(github_token=github_token))
|
||||
rate = json.loads(resp.content)
|
||||
print(f'token rate {rate.get("rate")}')
|
||||
except Exception as e:
|
||||
print(f'get rate limit failed: {e}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='params')
|
||||
parser.add_argument('--github_repository', nargs='?', type=str, required=True, help='github repository')
|
||||
@ -334,6 +343,9 @@ if __name__ == '__main__':
|
||||
if not args.github_repository:
|
||||
raise 'Need specific github repository'
|
||||
|
||||
if args.github_token:
|
||||
rate_limit(args.github_token)
|
||||
|
||||
try:
|
||||
if not os.path.isdir(args.cache_directory):
|
||||
os.makedirs(args.cache_directory, exist_ok=True)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user