minoplhy/filters-maker
1
0
Fork 0
Code Issues Pull Requests Projects Releases 2 Wiki Activity

Update 'crawler.py'

Browse Source
This commit is contained in:
Kale 2021-08-17 22:15:39 -07:00
parent 53607648cb
commit 5bbf6b4a72
No known key found for this signature in database
GPG Key ID: F8A6CE83F1274467
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
crawler.py
View File

@ -75,6 +75,11 @@ def filteringcon(filters_regex_one):
if not line.endswith((tuple(remove_words))):
f.write('\n'.join([line + '\n']))
f.close()
with open(filters_regex_one, 'r') as f:
lines = f.read().splitlines()
with open(filters_regex_one, 'w') as f:
for line in lines:
f.write('\n'.join(line.split())
def killingdup(duplicated_file):
print('Getting rid of duplicated line')