Method of working with regional ip address databases without using a firewall address list - does not take away the performance of executing rules using lists.
Usage example
Ready file with regional database: https://syo.su/download/MikroTikIpLocation.zip other https://syo.su/download/MikroTikIpMaxMind.zip
To run this file on your device, need about 80-100MB free of RAM. Unzip archive and download the file to device. Run the command:After rebooting the device, all variables will disappear, such a script run is required every time the device starts working. Save the usage example as a script, paste your external ip address into it and run it in the console.
In this way, you can identify unnecessary addresses that are currently accessing the device and place them in a dynamic ban list - the CPU load will be less than storing the regional ip addresses database on the firewall. This will also allow you to play around with large address databases without cluttering up the device’s memory. The database loaded in this way on the device takes up 2 times less RAM than if you load the same addresses into firewall address list. Searching for an address is faster than a standard search in firewall address list. Moreover, if you increase size of the database several times, for example, create a database with addresses linked by city, the speed of searching for an address in the database will practically not change, only more RAM will be required.
To assemble an array with a database of addresses yourself, download, for example, the CSV file with database DB1 from https://lite.ip2location.com/database-download
Go to page http://syo.su expand the section "Create subnets from CSV files with network addresses in any form", open downloaded file and set:
File delimiter: , (comma)
Files contains headers: uncheck
First or only addresses range column: 1
Second addresses range column: 2
Addresses format: integer
Addresses mask column: none
Files have joined data tables: uncheck
JavaScript expression of list names values:
JavaScript expression of comments values:
Header row value: not set
Merge ranges with same list name into the same subnet, if possible: check
Also group by comments: check
Set output file name
Limit file size: empty or 0
Select output file template: MikroTik search array
Click button Get file by template
Change output file name and select template: MikroTik search array addon
Click button Get file by template
The resulting files can be combined into one, or you can download and run separately in any order. You can also check the database for intersection of subnets - this method of searching for addresses assumes that all subnets in the database are unique. And you can get a list of subnets missed in the database.
Code:
:global ipLocation {"1.0.0.0"={24;"US"};"1.0.1.0"={24;"CN"};"1.0.2.0"={23;"CN"};"1.0.4.0"={22;"AU"};"1.0.8.0"={21;"CN"};"1.0.16.0"={20;"JP"};# . . . . . . . . .}:global ipCountries {"AU"="Australia";"CN"="China";"JP"="Japan";"US"="United States of America";# . . . . . . . . .}:global ipMinPrefix 32:global ipMaxPrefix 0:foreach param in=$ipLocation do={ :if (($param->0) < $ipMinPrefix) do={ :set ipMinPrefix ($param->0) } :if (($param->0) > $ipMaxPrefix) do={ :set ipMaxPrefix ($param->0) }}:global ipGetLocation do={ :global ipLocation :global ipMaxPrefix :global ipMinPrefix :local network [:toip $1] :local submask (255.255.255.255 << (32 - $ipMaxPrefix)) :for prefix from=$ipMaxPrefix to=$ipMinPrefix step=-1 do={ :local subnet [:tostr $network] :if ([:typeof ($ipLocation->$subnet)] != "nothing" && ($ipLocation->$subnet->0) <= $prefix) do={ :return ($ipLocation->$subnet->1) } :set submask ($submask << 1) :set network ($network & $submask) } :return []}:global ipGetSubnet do={ :global ipLocation :global ipMaxPrefix :global ipMinPrefix :local network [:toip $1] :local submask (255.255.255.255 << (32 - $ipMaxPrefix)) :for prefix from=$ipMaxPrefix to=$ipMinPrefix step=-1 do={ :local subnet [:tostr $network] :if ([:typeof ($ipLocation->$subnet)] != "nothing" && ($ipLocation->$subnet->0) <= $prefix) do={ :return ($subnet."/".($ipLocation->$subnet->0)) } :set submask ($submask << 1) :set network ($network & $submask) } :return []}:global ipGetInfo do={ :global ipLocation :global ipMaxPrefix :global ipMinPrefix :local network [:toip $1] :local submask (255.255.255.255 << (32 - $ipMaxPrefix)) :for prefix from=$ipMaxPrefix to=$ipMinPrefix step=-1 do={ :local subnet [:tostr $network] :if ([:typeof ($ipLocation->$subnet)] != "nothing" && ($ipLocation->$subnet->0) <= $prefix) do={ :return {($subnet."/".($ipLocation->$subnet->0)); ($ipLocation->$subnet->1)} } :set submask ($submask << 1) :set network ($network & $submask) } :return [:toarray ""]}Usage example
Code:
:global ipCountries:global ipGetLocation:local allConn [/ip firewall connection print detail as-value]:foreach conn in=$allConn do={ :local src [:pick ($conn->"src-address") 0 [:find ($conn->"src-address") ":"]] :local dst [:pick ($conn->"dst-address") 0 [:find ($conn->"dst-address") ":"]] :local srcWhois $src :local dstWhois $dst :if ($src = "Your external ip here") do={ :set srcWhois "Router" } else={ :local ip [:toip $src] :if ($ip & 255.0.0.0 = 10.0.0.0 || $ip & 255.255.0.0 = 192.168.0.0 || $ip & 255.240.0.0 = 172.16.0.0) do={ :set srcWhois "Localhost" } else={ :local ipCode [$ipGetLocation $src] :if ([:typeof $ipCode] != "nil") do={ :set srcWhois ($ipCountries->$ipCode) } } } :if ($dst = "Your external ip here") do={ :set dstWhois "Router" } else={ :local ip [:toip $dst] :if ($ip & 255.0.0.0 = 10.0.0.0 || $ip & 255.255.0.0 = 192.168.0.0 || $ip & 255.240.0.0 = 172.16.0.0) do={ :set dstWhois "Localhost" } else={ :local ipCode [$ipGetLocation $dst] :if ([:typeof $ipCode] != "nil") do={ :set dstWhois ($ipCountries->$ipCode) } } } :put ($srcWhois." -> ".$dstWhois)}# same thing using another function:global ipGetInfo:foreach conn in=$allConn do={ :local src [:pick ($conn->"src-address") 0 [:find ($conn->"src-address") ":"]] :local dst [:pick ($conn->"dst-address") 0 [:find ($conn->"dst-address") ":"]] :local srcWhois $src :local dstWhois $dst :if ($src = "Your external ip here") do={ :set srcWhois "Router" } else={ :local ip [:toip $src] :if ($ip & 255.0.0.0 = 10.0.0.0 || $ip & 255.255.0.0 = 192.168.0.0 || $ip & 255.240.0.0 = 172.16.0.0) do={ :set srcWhois "Localhost" } else={ :local ipInfo [$ipGetInfo $src] :if ([:len $ipInfo]) do={ :set srcWhois (($ipCountries->($ipInfo->1))." (".($ipInfo->0).")") } } } :if ($dst = "Your external ip here") do={ :set dstWhois "Router" } else={ :local ip [:toip $dst] :if ($ip & 255.0.0.0 = 10.0.0.0 || $ip & 255.255.0.0 = 192.168.0.0 || $ip & 255.240.0.0 = 172.16.0.0) do={ :set dstWhois "Localhost" } else={ :local ipInfo [$ipGetInfo $dst] :if ([:len $ipInfo]) do={ :set dstWhois (($ipCountries->($ipInfo->1))." (".($ipInfo->0).")") } } } :put ($srcWhois." -> ".$dstWhois)}Ready file with regional database: https://syo.su/download/MikroTikIpLocation.zip other https://syo.su/download/MikroTikIpMaxMind.zip
To run this file on your device, need about 80-100MB free of RAM. Unzip archive and download the file to device. Run the command:
Code:
import file=MikroTikIpLocation.rscIn this way, you can identify unnecessary addresses that are currently accessing the device and place them in a dynamic ban list - the CPU load will be less than storing the regional ip addresses database on the firewall. This will also allow you to play around with large address databases without cluttering up the device’s memory. The database loaded in this way on the device takes up 2 times less RAM than if you load the same addresses into firewall address list. Searching for an address is faster than a standard search in firewall address list. Moreover, if you increase size of the database several times, for example, create a database with addresses linked by city, the speed of searching for an address in the database will practically not change, only more RAM will be required.
To assemble an array with a database of addresses yourself, download, for example, the CSV file with database DB1 from https://lite.ip2location.com/database-download
Go to page http://syo.su expand the section "Create subnets from CSV files with network addresses in any form", open downloaded file and set:
File delimiter: , (comma)
Files contains headers: uncheck
First or only addresses range column: 1
Second addresses range column: 2
Addresses format: integer
Addresses mask column: none
Files have joined data tables: uncheck
JavaScript expression of list names values:
Code:
COLUMN[3] == '-' ? '' : COLUMN[3]Code:
COLUMN[4]Merge ranges with same list name into the same subnet, if possible: check
Also group by comments: check
Set output file name
Limit file size: empty or 0
Select output file template: MikroTik search array
Click button Get file by template
Change output file name and select template: MikroTik search array addon
Click button Get file by template
The resulting files can be combined into one, or you can download and run separately in any order. You can also check the database for intersection of subnets - this method of searching for addresses assumes that all subnets in the database are unique. And you can get a list of subnets missed in the database.
Statistics: Posted by DenSyo77 — Tue Jan 23, 2024 4:43 am