IPTables do not block IP with ipset immediately

I have the following IPTables with IPSet as rule source to block attacking IP, but when I add an attacking IP to IPSet, in my nginx access log, I still see continuous access of the attack IP. After a while，maybe 3~5 minutes, the IP was blocked.

iptables

~$ sudo iptables -nvL --line-numbers
Chain INPUT (policy ACCEPT 317K packets, 230M bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     106K 6004K DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set Blacklist src

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set Blacklist src

Chain OUTPUT (policy ACCEPT 350K packets, 58M bytes)
num   pkts bytes target     prot opt in     out     source               destination

ipset

sudo ipset -L
Name: Blacklist
Type: hash:ip
Revision: 4
Header: family inet hashsize 1024 maxelem 65536 timeout 60
Size in memory: 13280
References: 2
Members:
xxx.xxx.xxx.xxx(attacker ip) timeout 0

I don't know why the rule has not effect immediately, which make me crazy just like the attacker is laughing at me.

I add ipset to the iptables rule with -I option which should keep the rule at the first position. So maybe the Chain INPUT(policy Accept) do the trick?

Please help me out, thanks so much.

BTW.

I use Nginx+Djano/uWSGI to deploy my application, and I use shell script to analyze nginx log to put evil ip to Blacklist ipset.