Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Are you looking for help with adding/updating lists and metadata? #1674

Open
DandelionSprout opened this issue Jun 9, 2020 · 9 comments
Open
Labels
announcement community updates directory-data changes to basic FilterLists data

Comments

@DandelionSprout
Copy link
Contributor

DandelionSprout commented Jun 9, 2020

(This issue report will be a carry-over of #1663 (comment) and #851, with the intention that it shall be stickied, perma-open, and linked to in the README, to help and inspire more people to contribute to the metadata.)

So you know of some lists you want to add, but you are very unsure about how to do so? Fear no more. Here's how to do it:

Add/update lists, licenses, subscription links, homepages, and similar:

  1. Right-click on the following link and choose "Save link": https://raw.githubusercontent.com/collinbarrett/FilterLists/master/services/Directory/data/FilterList.json
  2. Open the downloaded FilterList.json file in Sublime Text, Notepad++, or any similar raw-text editor who can handle large files and JSON indentations.
  3. Add the desired changes. For new lists, "id", "name", "syntaxId" and "viewUrl" are mandatory. "homeUrl" and "licenseId" are also important if they are applicable to the list. Other ones in active use are "description", "donateUrl", "emailAddress", "issuesUrl", "viewUrlMirror1" and "viewUrlMirror2".
  4. Once you've performed the changes, save the file.
  5. Before you can upload the changes to this repo, you must first create a branch of this repo. The easiest way to do so by far, is to take care of assigning tags to the list first (Explained below).
  6. Once you're on a new branch, go to https://github.com/[Your GitHub username]/FilterLists/tree/master/services/Directory/data (The URL may be different under many circumstances, such as if you've contributed to FilterLists.com before), click "Upload" in the upper right, and choose the FilterList.json file that you saved.
  7. Submit a PR, and we'll double-check it from there.

Assign tags to lists (Mandatory for new lists):

  1. Go to https://github.com/collinbarrett/FilterLists/edit/master/services/Directory/data/FilterListTag.json, and edit it online.
  2. "filterListId" is the list's ID in FilterList.json. "tagId" is one of these tags.
  3. If you are 100% certain that none of the existing tags are suitable for a particular list, choose tagId 15 (Topical).
  4. Save the file. This will create a new branch and/or fork, which will be convenient for list additions in general.

Assign languages to lists:

(This presumes you've already assigned tags to the lists.)

  1. Go to the branch you created after assigning tags.
  2. Go to .../data/FilterListLanguage.json and begin editing it.
  3. "languageId" is one of a randomly numbered list of languages. For instance, German is 167, while Japanese is 141.
  4. Save the file.

Assign contributors to lists:

(This presumes you've already assigned tags to the lists.)

  1. Go to the branch you created after assigning tags.
  2. Go to .../data/FilterListMaintainer.json and begin editing it.
  3. "maintainerId" is one of various filterlist makers. If you wish to assign someone who is not mentioned in that file, you are merrily welcome to add maintainer profiles to Maintainer.json
  4. Save.

Moreover, this thread can also be used to store your backlogs of things to do, and for whom help is requested to get it done faster.

@DandelionSprout
Copy link
Contributor Author

DandelionSprout commented Jun 9, 2020

Things I've come across. Anyone else out there are welcome to write a comment below to store things that they have come across. If possible, edit your existing to-do list instead of creating additional comments each time.

Add some of the following lists:

Add mirrors:

(Currently empty.)

Change viewURLs:

(Currently empty.)

Change syntax:

(Currently empty.)

Change title:

(Currently empty.)

Remove lists:

(Currently empty.)

Remove view URLs:

Notes to self when deleting lists:

(Replace "9876" with each of the filterlist IDs separated by |.)

  • FilterList.json: .* {\n(.*\n){1,7} {1,}"id": (9876),(.*\n){1,6} {1,}},\n
  • FilterListTags.json + FilterListViewUrl.json + FilterListSyntax.json + FilterListLanguage.json + FilterListMaintainer.json: .* {\n {1,}"filterListId": (9876),(.*\n){1,4} {1,}},\n

@collinbarrett
Copy link
Owner

Nice. Great work. Maybe will move to a Wiki page eventually rather than keeping in an Issue long-term, but will prominently link to from README as well.

@ftpmorph
Copy link
Contributor

ftpmorph commented Jun 28, 2020

Thank you for the detailed guide, the readme on the initial page of the project had me at a bit worried this would take ages. Once I read this and worked out it was just a case of forking the repo and editing a few json files I was much more comfortable.

Now that I said that, I hope I haven't messed anything up... would look a bit silly if I did.

@ambuj-codes
Copy link

https://www.pnas.org is a false entry in porn hostlist.
https://www.pnas.org is not a porn site.

@collinbarrett
Copy link
Owner

@ambuj-codes You will need to contact the maintainer of whichever list contains the false positives directly. The FilterLists project itself does not maintain any lists. We just provide an index of known lists and info about how to contact the maintainers, etc.

@collinbarrett
Copy link
Owner

collinbarrett commented Sep 7, 2020

I just completed a somewhat significant refactor of the data model. Some of the links in the OP are now broken.

  • Data now lives here.
  • The TPL list has moved here (so that it can be included on filterlists.com here).
  • The Wiki has been updated to match the new version of the data.

@collinbarrett
Copy link
Owner

Thanks, @rimu . Looks like you figured it out, but I just edited some of the urls in the OP since the project paths have changed a bit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
announcement community updates directory-data changes to basic FilterLists data
Projects
None yet
Development

No branches or pull requests

6 participants