Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
42916f2
changed grep command to work with busybox
Br1cked Sep 25, 2014
59b8924
update blacklist atomically with `ipset swap`
Br1cked Sep 25, 2014
d6a3729
change curl to wget
Br1cked Sep 25, 2014
a0bc106
removed unused variable
Br1cked Sep 25, 2014
f98e973
fixed grep and sort bugs
Br1cked Sep 25, 2014
922d520
updated tor exit node list URL
Br1cked Sep 25, 2014
0ab89bb
removed 404 URL
Br1cked Sep 25, 2014
0fd6099
replaced 2 blacklist URLs with a combo URL
Br1cked Sep 25, 2014
de73092
added sed command to parse emergingthreats.net lists
Br1cked Sep 25, 2014
3526662
Update update-blacklist.sh
Br1cked Sep 25, 2014
d132470
added 2 URL removed in previous commits
Br1cked Sep 25, 2014
7283757
busybox (ash) compatibility & memory optimizations
Br1cked Oct 1, 2014
faaba73
Update README.md
Br1cked Oct 1, 2014
d94b49a
refactored
Br1cked Oct 2, 2014
16c7a7c
added ipset maxelem option & more blacklist URLs
Br1cked Oct 8, 2014
8dc5195
syntax bug fix
Br1cked Oct 8, 2014
bb7eec4
syntax bug fix
Br1cked Oct 8, 2014
4b7eb01
bug fix
Br1cked Oct 8, 2014
f2d2205
major update/refactor to handle ranges & cidr notation
Br1cked Oct 8, 2014
1f2c6d0
added lua script to remove overlapping cidr ranges
Oct 17, 2014
db32bd6
uncommented lines
Oct 17, 2014
45854fc
created load_blacklist script and added a url
Oct 17, 2014
e7b1ce0
updated readme
Oct 17, 2014
518b036
updated readme and misc changes
Oct 17, 2014
a060a8c
uncommented lines (commented for debugging)
Oct 17, 2014
346698c
chmod +x
Oct 17, 2014
f5185ca
call lua script directly (must be on PATH)
Oct 17, 2014
245eb61
corrected install path for bindechex.lua
Br1cked Oct 17, 2014
3daf634
added checks for failed commands
Oct 18, 2014
bd46b67
updated for clarity
Br1cked Oct 18, 2014
f61d2f2
bug fix
Nov 12, 2014
6887107
added install script
Nov 12, 2014
6fb50b5
updated README
Nov 12, 2014
2b42144
minor bug fix
Nov 12, 2014
b152334
added firewall rules to load_blacklist.sh script
Nov 12, 2014
cc9d531
updated install script
Nov 12, 2014
e054560
Update README.md
Br1cked Nov 12, 2014
0d49824
added lua to required packages, and removed ionice
Br1cked Jan 3, 2015
da2d764
removed test code
Br1cked Jan 3, 2015
98c41de
fixed synchronization bug
Br1cked Jan 3, 2015
f527730
print error messages to stdout
Br1cked Jan 3, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 10 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,27 @@
ipset-blacklist
===============

A tiny Bash shell script which uses ipset and iptables to ban a large number of IP addresses published in IP blacklists. ipset uses a hashtable to store/fetch IP addresses and thus the IP lookup is a lot (!) faster than thousands of sequentially parsed iptables ban rules. However, the limit of an ipset list is 2^16 entries.
A set of shell scripts which use ipset and iptables to ban a large number of IP addresses published in IP blacklists. `ipset` uses a hashtable to store/fetch IP addresses and thus the IP lookup is a lot (!) faster than thousands of sequentially parsed iptables ban rules. However, the limit of an ipset list is 2^16 entries.

The ipset command doesn't work under OpenVZ. It works fine on dedicated and fully virtualized servers like KVM though.
Note: Updating the blacklists takes a long time (8m 20s on a TPLink TL-WDR3600), and running at a low priority (with `nice` and `ionice`) is recommended to avoid impacting packet routing & firewall performance.

## Quick start for Debian/Ubuntu based installations
1. Copy update-blacklist.sh into /usr/local/bin
2. chmod +x /usr/local/bin/update-blacklist.sh
2. Modify update-blacklist.sh according to your needs. Per default, the blacklisted IP addresses will be saved to /etc/ip-blacklist.conf
3. apt-get install ipset
4. Create the ipset blacklist and insert it into your iptables input filter (see below). After proper testing, make sure to persist it in your firewall script or similar or the rules will be lost after the next reboot.
5. Auto-update the blacklist using a cron job
## Quick start for OpenWRT
1. Copy the scripts to a temporary directory on your OpenWRT router
2. Run `sh install.sh`

# iptables filter rule
```
ipset create blacklist hash:net
iptables -I INPUT -m set --match-set blacklist src -j DROP
ipset create blacklist_net hash:net
ipset create blacklist_ip hash:ip
iptables -I INPUT -m set --match-set blacklist_net src -j DROP
iptables -I INPUT -m set --match-set blacklist_ip src -j DROP
```
Make sure to run this snippet in your firewall script. If you don't, the ipset blacklist and the iptables rule to ban the blacklisted ip addresses will be missing!

# Cron job
In order to auto-update the blacklist, copy the following code into /etc/cron.d/update-blacklist. Don't update the list too often or some blacklist providers will ban your IP address. Once a day should be OK though.
```
MAILTO=root
33 23 * * * root /usr/local/bin/update-blacklist.sh
33 23 * * * /bin/nice /usr/local/bin/update-blacklist.sh > /tmp/update-blacklist.log 2>&1
```

## Check for dropped packets
Expand All @@ -36,13 +33,3 @@ Chain INPUT (policy DROP 3064 packets, 177K bytes)
pkts bytes target prot opt in out source destination
43 2498 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set blacklist src
```

## Modify the blacklists you want to use
Edit the BLACKLIST array to add or remove blacklists, or use it to add your own blacklists.
```
BLACKLISTS=(
"http://www.mysite.me/files/mycustomblacklist.txt" # Your personal blacklist
"http://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1" # Project Honey Pot Directory of Dictionary Attacker IPs
# I don't want this: "http://www.openbl.org/lists/base.txt" # OpenBL.org 30 day List
)
```
Loading