标题: [文本处理] 【已解决】紧急求助批处理IIS日志多文件处理,万分感谢! [打印本页]
作者: shniutan 时间: 2022-4-21 00:17 标题: 【已解决】紧急求助批处理IIS日志多文件处理,万分感谢!
本帖最后由 shniutan 于 2022-5-5 08:43 编辑
非常感谢idwma,qixiaobin0715,xczxczxcz 三位大神的无私帮助,谢谢你们!
有3个IIS日志文件,如下:
a.txt //每行包含1个infoID,各不相同
2022-1-2 08:00:01 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80
2022-1-2 08:00:06 192.168.1.10 post \file\detail.aspx\infoid=1q1w3e4r-6y2u8i 80
2022-1-2 08:00:11 192.168.1.10 post \file\detail.aspx\infoid=1q2w4e4r-6y3u8i 80
2022-1-2 08:00:16 192.168.1.10 post \file\detail.aspx\infoid=1q3w3e4r-6y4u8i 80
2022-1-2 08:00:21 192.168.1.10 post \file\detail.aspx\infoid=1q4w3e4r-6y5u8i 80
2022-1-2 08:00:26 192.168.1.10 post \file\detail.aspx\infoid=1q5w3e4r-6y6u8i 80
2022-1-2 08:00:31 192.168.1.10 post \file\detail.aspx\infoid=1q6w3e4r-6y7u8i 80
2022-1-2 08:00:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80
2022-1-2 08:00:41 192.168.1.10 post \file\detail.aspx\infoid=1q8w3e4r-6y9u8i 80
2022-1-2 08:00:46 192.168.1.10 post \file\detail.aspx\infoid=2q2w3e4r-6y1u8i 80
b.txt //每行包含用户名、IP地址,有用户名、IP地址均重复的行
2022-1-2 08:00:05 192.168.1.10 get \bbs\login.aspx\username=张三&password=6y1u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:00:15 192.168.1.10 get \bbs\login.aspx\username=李四关&password=6y1u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:00:25 192.168.1.10 get \bbs\login.aspx\username=王五&password=6y1u8i 80 - 192.168.1.25 200 0 0
2022-1-2 08:00:35 192.168.1.10 get \bbs\login.aspx\username=孙六&password=6y1u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:00:45 192.168.1.10 get \bbs\login.aspx\username=刘七地&password=6y1u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:00:55 192.168.1.10 get \bbs\login.aspx\username=杨八猫&password=6y1u8i 80 - 192.168.1.31 200 0 0
2022-1-2 08:01:05 192.168.1.10 get \bbs\login.aspx\username=江九&password=6y1u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:03:25 192.168.1.10 get \bbs\login.aspx\username=周十&password=6y1u8i 80 - 192.168.1.35 200 0 0
2022-1-2 08:03:45 192.168.1.10 get \bbs\login.aspx\username=巴一巴&password=6y1u8i 80 - 192.168.1.37 200 0 0
2022-1-2 08:04:25 192.168.1.10 get \bbs\login.aspx\username=赵二&password=6y1u8i 80 - 192.168.1.39 200 0 0
2022-1-2 09:00:25 192.168.1.10 get \bbs\login.aspx\username=王五&password=6y1u8i 80 - 192.168.1.25 200 0 0
2022-1-2 09:00:35 192.168.1.10 get \bbs\login.aspx\username=孙六&password=6y1u8i 80 - 192.168.1.27 200 0 0
2022-1-2 09:00:45 192.168.1.10 get \bbs\login.aspx\username=刘七地&password=6y1u8i 80 - 192.168.1.29 200 0 0
2022-1-2 09:00:55 192.168.1.10 get \bbs\login.aspx\username=杨八猫&password=6y1u8i 80 - 192.168.1.31 200 0 0
c.txt //每行包含文件ID、IP地址,存在同一个IP访问不同文件ID的情况,也存在不同IP访问同一文件的情况
2022-1-2 08:00:01 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:00:02 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:00:03 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.25 200 0 0
2022-1-2 08:00:04 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:00:06 192.168.1.10 post \file\detail.aspx\infoid=1q1w3e4r-6y2u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:00:11 192.168.1.10 post \file\detail.aspx\infoid=1q2w4e4r-6y3u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:00:16 192.168.1.10 post \file\detail.aspx\infoid=1q3w3e4r-6y4u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:00:21 192.168.1.10 post \file\detail.aspx\infoid=1q4w3e4r-6y5u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:00:26 192.168.1.10 post \file\detail.aspx\infoid=1q5w3e4r-6y6u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:00:31 192.168.1.10 post \file\detail.aspx\infoid=1q6w3e4r-6y7u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:00:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:00:41 192.168.1.10 post \file\detail.aspx\infoid=1q8w3e4r-6y9u8i 80 - 192.168.1.35 200 0 0
2022-1-2 08:00:46 192.168.1.10 post \file\detail.aspx\infoid=2q2w3e4r-6y1u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:00:51 192.168.1.10 post \file\detail.aspx\infoid=3q2w3e4r-6y1u8i 80 - 192.168.1.37 200 0 0
2022-1-2 08:00:56 192.168.1.10 post \file\detail.aspx\infoid=4q2w3e4r-6y7u8i 80 - 192.168.1.35 200 0 0
2022-1-2 08:01:16 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:01:51 192.168.1.10 post \file\detail.aspx\infoid=5q2w3e4r-6y7u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:01:56 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.25 200 0 0
2022-1-2 08:01:26 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:01:36 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:01:46 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.25 200 0 0
2022-1-2 08:02:21 192.168.1.10 post \file\detail.aspx\infoid=7q2w3e4r-6y7u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:03:36 192.168.1.10 post \file\detail.aspx\infoid=8q2w3e4r-6y7u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:10:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80 - 192.168.1.31 200 0 0
2022-1-2 08:20:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80 - 192.168.1.31 200 0 0
2022-1-2 08:30:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80 - 192.168.1.31 200 0 0
2022-1-2 08:33:46 192.168.1.10 post \file\detail.aspx\infoid=2q2w3e4r-6y1u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:34:21 192.168.1.10 post \file\detail.aspx\infoid=1q4w3e4r-6y5u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:34:26 192.168.1.10 post \file\detail.aspx\infoid=1q5w3e4r-6y6u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:40:01 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:40:02 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:40:03 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.25 200 0 0
希望能用批处理实现:
1.删除b.txt中用户名、IP地址均重复的行,输出结果到d.txt。
2.从c.txt中找出a.txt中每个infoID的被访问情况,分别输出到e1.txt、e2.txt、……中,再根据d.txt中用户名和IP地址的对应关系,统计出e1.txt、e2.txt、……中每个IP地址对应用户的访问次数,分别输出infoid、IP地址、用户名、访问次数信息到f1.txt、f2.txt、……中。
3.查找f1.txt、f2.txt、……所有文件中infoid被同一用户访问超过2次的记录,汇总输出到新的文件中,每个用户1个文件,输出信息包括IP地址、用户名、infoid、访问次数。
万分感谢!
作者: idwma 时间: 2022-4-21 20:18
假设def的文件都是多余的设只要结果- #@&cls&powershell "type %~s0|out-string|iex"&pause&exit
- $a=@{}
- gc b.txt|%{$a[$_ -replace '.*name=(.*)&.*-\s(\S+).*','$1 $2']=$_}
- $b=(gc .\c.txt) -match (((gc a.txt) -replace '.*infoid=(\S+).*','$1') -join '|')
- $c=@{}
- $a.keys|%{$t=$_ -split '\s+'; $c[$t[1]]=$t[0]}
- $d=@{}
- $b|%{$t=$_ -replace '.*infoid=([\S]+).*-\s([\d\.]+).*','$1 $2';$d[$t]+=1}
- $e=$d.keys|%{$t=$_ -split '\s+';"{0} {1}`t{2} {3}" -f $t[1],$c[$t[1]],$t[0],$d[$_]}
- $f=@{}
- $e|%{$t=$_ -split '\s+';$f[$t[1]]+=[int]$t[3]}
- $g=($f.keys|?{$f[$_] -gt 2}) -join '|'
- $h=@{}
- $e -match $g|%{$t=$_ -split '\s+';$h[$t[1]]+=@($_)}
- $h.keys|%{sc "$_.txt" $h[$_]}
复制代码
作者: qixiaobin0715 时间: 2022-4-22 14:28
回复 1# shniutan
描述太乱,写代码时也改来改去很乱,不知这样汇总是不是你要的结果:- @echo off
- setlocal enabledelayedexpansion
- for /f "tokens=8,12 delims=-&= " %%i in (b.txt) do set #%%j=%%i
- (for /f "tokens=1* delims==" %%x in (a.txt) do echo,%%y)>1.log
- findstr /g:1.log c.txt>Newfile.txt
- (for /f "tokens=8,9,11 delims=-= " %%a in (Newfile.txt) do (
- set /a "_%%a%%b+=1"
- if "!_%%a%%b!" == "2" echo,%%a-%%b
- ))>1.log
- findstr /g:1.log Newfile.txt>2.log
- (for /f "tokens=8,9,11 delims=-= " %%a in (2.log) do (
- if not defined @%%a%%b%%c echo,%%c !#%%c! %%a-%%b !_%%a%%b!
- set @%%a%%b%%c=true
- ))>1.log
- sort 1.log>Newfile.txt
- del *.log
- pause
复制代码
作者: shniutan 时间: 2022-4-22 18:54
@idwma,@qixiaobin0715
非常感谢两位大佬帮忙!
抱歉之前没有表述清楚。又重新梳理一下需求:
1.不需要删除b.txt中IP地址和人名重复的行这步操作,b.txt文件作用就是通过IP地址找对对应人名。
2.在c.txt中找出每个IP地址访问的infoid(仅统计a.txt中列出的)次数,并根据b.txt中IP地址和人名的对应关系,输出相关信息到1个文件中。
最终输出的结果应该是:
192.168.1.21 张三 1q2w3e4r-6y1u8i 2
192.168.1.21 张三 1q2w4e4r-6y3u8i 1
192.168.1.23 李四关 1q2w3e4r-6y1u8i 2
192.168.1.23 李四关 1q1w3e4r-6y2u8i 1
192.168.1.23 李四关 1q4w3e4r-6y5u8i 2
192.168.1.25 王五 1q2w3e4r-6y1u8i 2
192.168.1.27 孙六 1q2w3e4r-6y1u8i 1
192.168.1.27 孙六 1q5w3e4r-6y6u8i 2
192.168.1.29 刘七地 1q3w3e4r-6y4u8i 1
192.168.1.29 刘七地 1q6w3e4r-6y7u8i 1
192.168.1.31 杨八猫 1q7w3e4r-6y8u8i 3
192.168.1.33 江九 1q7w3e4r-6y8u8i 1
192.168.1.33 江九 2q2w3e4r-6y1u8i 2
192.168.1.35 周十 1q8w3e4r-6y9u8i 1
//总次数应该是a.txt中infoid在c.txt中出现的总次数,22次。
作者: idwma 时间: 2022-4-22 19:03
本帖最后由 idwma 于 2022-4-22 19:07 编辑
- #@&cls&powershell "type %~s0|out-string|iex"&pause&exit
- $a=@{}
- gc b.txt|%{$a[$_ -replace '.*name=(.*)&.*-\s(\S+).*','$1 $2']=$_}
- $b=(gc .\c.txt) -match (((gc a.txt) -replace '.*infoid=(\S+).*','$1') -join '|')
- $c=@{}
- $a.keys|%{$t=$_ -split '\s+'; $c[$t[1]]=$t[0]}
- $d=@{}
- $b|%{$t=$_ -replace '.*infoid=([\S]+).*-\s([\d\.]+).*','$1 $2';$d[$t]+=1}
- $e=$d.keys|%{$t=$_ -split '\s+';"{0} {1}`t{2} {3}" -f $t[1],$c[$t[1]],$t[0],$d[$_]}
- $f=@{}
- $e|%{$t=$_ -split '\s+';$f[$t[1]]+=[int]$t[3]}
- $g=($f.keys) -join '|'
- $h=@{}
- $e -match $g|%{$t=$_ -split '\s+';$h[$t[1]]+=@($_)}
- $h.values|sc abc.txt
复制代码
作者: shniutan 时间: 2022-4-23 22:21
本帖最后由 shniutan 于 2022-4-24 10:26 编辑
回复 5# idwma
非常感谢百忙之中帮我解决问题!
a.txt
2010/12/23 05:38:35 inforID=AAF5AB44-389D-44AD-84EA-A45C6EE74AFE
2011/01/18 01:06:33 inforID=7717F5D7-A409-4D48-9E87-1A92367C3593
2011/11/15 01:37:00 inforID=E9D8DE66-A11E-4778-8CEC-64C46B657E6D
2012/02/14 02:53:43 inforID=580552F2-4A22-41FB-B5A8-8E4806E90C08
2013/05/07 06:23:04 inforID=99D32186-121E-4FF7-AB54-0F816687A0A5
2014/08/06 06:12:07 inforID=1F698943-4766-4C48-ADB5-256ED89E1AF8
2015/07/21 05:23:39 inforID=C8FC9543-453E-44AB-9188-FC72963689D1
2017/08/02 11:20:26 inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE
2021/10/25 09:28:17 inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299
2022/04/02 10:04:52 inforID=C049F1E8-CA33-436B-9F67-45E5A8E792D5
b.txt //因系统原因,2013年前的日志包含了浏览器信息
2010-12-16 04:46:00 192.168.10.1 GET /bbs/LoginForum.aspx UserName=工程部_张三a&Password=qaeswdrf 80 - 192.168.37.153 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727) 200 0 0
2010-12-26 04:46:33 192.168.10.1 GET /bbs/LoginForum.aspx UserName=服务部_李四博a&Password=qaeswdrf 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+InfoPath.1;+.NET+CLR+1.1.4322) 200 0 0
2011-12-06 04:47:05 192.168.10.1 GET /bbs/LoginForum.aspx UserName=投诉部_王欣零a&Password=qaeswdrf 80 - 192.168.81.18 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2012-02-16 04:47:43 192.168.10.1 GET /bbs/LoginForum.aspx UserName=人事部_江五a&Password=qaeswdrf 80 - 192.168.29.193 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.2;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.5.21022;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) 200 0 0
2013-10-16 04:48:23 192.168.10.1 GET /bbs/LoginForum.aspx UserName=技术部_郭肥来a&Password=qaeswdrf 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) 200 0 0
2014-11-04 01:28:36 192.168.10.1 GET /bbs/LoginForum.aspx UserName=工程部_赵六华a&Password=qaeswdrf 80 - 192.168.37.136 200 0 0
2015-01-17 01:28:53 192.168.10.1 GET /bbs/LoginForum.aspx UserName=服务部_周小宁a&Password=qaeswdrf 80 - 192.168.102.31 200 0 0
2016-01-08 01:29:04 192.168.10.1 GET /bbs/LoginForum.aspx UserName=人事部_肖逄文a&Password=qaeswdrf 80 - 192.168.29.10 200 0 0
2018-02-04 01:29:26 192.168.10.1 GET /bbs/LoginForum.aspx UserName=投诉部_刘瑾a&Password=qaeswdrf 80 - 192.168.81.31 200 0 0
2021-10-04 01:29:57 192.168.10.1 GET /bbs/LoginForum.aspx UserName=技术部_龙一a&Password=qaeswdrf 80 - 192.168.30.20 200 0 0
c.txt //因系统原因,2013年前的日志包含了浏览器信息
2011-01-18 03:33:09 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.20 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+Maxthon;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-01-18 03:36:09 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727) 200 0 0
2011-01-18 01:20:04 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1) 200 0 0
2011-01-19 06:59:55 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 07:10:04 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.29.10 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 07:10:11 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2) 200 0 0
2011-11-15 07:10:15 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 08:41:46 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.30.20 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+Maxthon;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2) 200 0 0
2013-05-07 06:23:11 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.29.10 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1;+InfoPath.2) 200 0 0
2013-05-07 06:24:23 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2013-05-07 06:25:12 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1;+InfoPath.2) 200 0 0
2013-05-07 06:28:22 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121B-4FF7-AB54-0F816687A0A5 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2;+TheWorld) 200 0 0
2013-05-07 06:26:05 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.37.153 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30) 200 0 0
2015-07-21 05:31:13 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.37.153 200 0 0
2015-07-21 05:31:30 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.102.11 200 0 0
2015-07-21 05:32:35 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.30.20 200 0 0
2015-07-21 05:33:21 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.37.153 200 0 0
2017-08-02 23:39:06 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.81.18 200 0 0
2017-08-02 23:39:31 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.30.20 200 0 0
2017-08-02 23:49:21 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.30.20 200 0 0
2017-08-02 23:53:54 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.102.11 200 0 0
2021-10-25 09:36:34 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.29.10 200 0 0
2021-10-25 09:36:44 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.30.1 200 0 0
2021-10-25 09:37:04 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.30.20 200 0 0
2021-10-25 09:37:50 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.37.153 200 0 0
2021-10-25 09:41:07 192.168.10.1 POST /File/Detail.aspx inforID=A0C8CDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.29.10 200 0 0
运行后报错,能输出结果文件,但输出的结果中多统计了2条未包含在a.txt中文件的访问记录。报错信息如下:
索引操作失败;数组索引的计算结果为空值。
所在位置 行:6 字符: 38
+ $a.keys|%{$t=$_ -split '\s+'; $c[ <<<< $t[1]]=$t[0]}
+ CategoryInfo : InvalidOperation: (System.Collections.Hashtable:
Hashtable) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
作者: qixiaobin0715 时间: 2022-4-24 09:32
回复 4# shniutan
加上顶楼的限定条件“所有文件中infoid被同一用户访问超过2次的记录”,总次数应当是17次:- @echo off
- setlocal enabledelayedexpansion
- for /f "tokens=8,12 delims=-&= " %%i in (b.txt) do set #%%j=%%i
- (for /f "tokens=1* delims==" %%x in (a.txt) do echo,%%y)>1.log
- findstr /g:1.log c.txt>Newfile.txt
- (for /f "tokens=8,9,11 delims=-= " %%a in (Newfile.txt) do (
- set /a "_%%a%%b+=1"
- set /a "_%%a%%b%%c+=1"
- if "!_%%a%%b!" == "2" echo,%%a-%%b
- ))>1.log
- findstr /g:1.log Newfile.txt>2.log
- (for /f "tokens=8,9,11 delims=-= " %%a in (2.log) do (
- if not defined @%%a%%b%%c (
- echo,%%c !#%%c! %%a-%%b !_%%a%%b%%c!
- set /a Sum+=!_%%a%%b%%c!
- )
- set @%%a%%b%%c=true
- ))>1.log
- sort 1.log>Newfile.txt
- echo,Sum !Sum!>>Newfile.txt
- del *.log
- pause
复制代码
作者: shniutan 时间: 2022-4-24 10:38
回复 7# qixiaobin0715
非常感谢!
用您的代码试了一下下面的文件,好像出不来结果。另外,不再限定访问次数为2以上了。
最终想实现的就是:
在c.txt中找出每个IP地址访问的inforID(仅统计a.txt中列出的)次数,并根据b.txt中IP地址和人名的对应关系,输出IP地址、人名、inforID、访问次数信息到1个文件中。
a.txt
2010/12/23 05:38:35 inforID=AAF5AB44-389D-44AD-84EA-A45C6EE74AFE
2011/01/18 01:06:33 inforID=7717F5D7-A409-4D48-9E87-1A92367C3593
2011/11/15 01:37:00 inforID=E9D8DE66-A11E-4778-8CEC-64C46B657E6D
2012/02/14 02:53:43 inforID=580552F2-4A22-41FB-B5A8-8E4806E90C08
2013/05/07 06:23:04 inforID=99D32186-121E-4FF7-AB54-0F816687A0A5
2014/08/06 06:12:07 inforID=1F698943-4766-4C48-ADB5-256ED89E1AF8
2015/07/21 05:23:39 inforID=C8FC9543-453E-44AB-9188-FC72963689D1
2017/08/02 11:20:26 inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE
2021/10/25 09:28:17 inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299
2022/04/02 10:04:52 inforID=C049F1E8-CA33-436B-9F67-45E5A8E792D5
b.txt //因系统原因,2013年前的日志包含了浏览器信息
2010-12-16 04:46:00 192.168.10.1 GET /bbs/LoginForum.aspx UserName=工程部_张三a&Password=qaeswdrf 80 - 192.168.37.153 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727) 200 0 0
2010-12-26 04:46:33 192.168.10.1 GET /bbs/LoginForum.aspx UserName=服务部_李四博a&Password=qaeswdrf 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+InfoPath.1;+.NET+CLR+1.1.4322) 200 0 0
2011-12-06 04:47:05 192.168.10.1 GET /bbs/LoginForum.aspx UserName=投诉部_王欣零a&Password=qaeswdrf 80 - 192.168.81.18 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2012-02-16 04:47:43 192.168.10.1 GET /bbs/LoginForum.aspx UserName=人事部_江五a&Password=qaeswdrf 80 - 192.168.29.193 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.2;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.5.21022;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) 200 0 0
2013-10-16 04:48:23 192.168.10.1 GET /bbs/LoginForum.aspx UserName=技术部_郭肥来a&Password=qaeswdrf 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) 200 0 0
2014-11-04 01:28:36 192.168.10.1 GET /bbs/LoginForum.aspx UserName=工程部_赵六华a&Password=qaeswdrf 80 - 192.168.37.136 200 0 0
2015-01-17 01:28:53 192.168.10.1 GET /bbs/LoginForum.aspx UserName=服务部_周小宁a&Password=qaeswdrf 80 - 192.168.102.31 200 0 0
2016-01-08 01:29:04 192.168.10.1 GET /bbs/LoginForum.aspx UserName=人事部_肖逄文a&Password=qaeswdrf 80 - 192.168.29.10 200 0 0
2018-02-04 01:29:26 192.168.10.1 GET /bbs/LoginForum.aspx UserName=投诉部_刘瑾a&Password=qaeswdrf 80 - 192.168.81.31 200 0 0
2021-10-04 01:29:57 192.168.10.1 GET /bbs/LoginForum.aspx UserName=技术部_龙一a&Password=qaeswdrf 80 - 192.168.30.20 200 0 0
c.txt //因系统原因,2013年前的日志包含了浏览器信息
2011-01-18 03:33:09 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.20 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+Maxthon;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-01-18 03:36:09 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727) 200 0 0
2011-01-18 01:20:04 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1) 200 0 0
2011-01-19 06:59:55 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 07:10:04 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.29.10 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 07:10:11 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2) 200 0 0
2011-11-15 07:10:15 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 08:41:46 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.30.20 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+Maxthon;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2) 200 0 0
2013-05-07 06:23:11 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.29.10 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1;+InfoPath.2) 200 0 0
2013-05-07 06:24:23 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2013-05-07 06:25:12 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1;+InfoPath.2) 200 0 0
2013-05-07 06:28:22 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121B-4FF7-AB54-0F816687A0A5 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2;+TheWorld) 200 0 0
2013-05-07 06:26:05 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.37.153 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30) 200 0 0
2015-07-21 05:31:13 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.37.153 200 0 0
2015-07-21 05:31:30 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.102.11 200 0 0
2015-07-21 05:32:35 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.30.20 200 0 0
2015-07-21 05:33:21 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.37.153 200 0 0
2017-08-02 23:39:06 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.81.18 200 0 0
2017-08-02 23:39:31 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.30.20 200 0 0
2017-08-02 23:49:21 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.30.20 200 0 0
2017-08-02 23:53:54 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.102.11 200 0 0
2021-10-25 09:36:34 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.29.10 200 0 0
2021-10-25 09:36:44 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.30.1 200 0 0
2021-10-25 09:37:04 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.30.20 200 0 0
2021-10-25 09:37:50 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.37.153 200 0 0
2021-10-25 09:41:07 192.168.10.1 POST /File/Detail.aspx inforID=A0C8CDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.29.10 200 0 0
作者: qixiaobin0715 时间: 2022-4-24 11:06
本帖最后由 qixiaobin0715 于 2022-4-24 11:21 编辑
回复 8# shniutan
你把a、b、c文本文件传到网盘上,可帮你测试。
你顶楼的文本好像与你实际文本并不相同。
作者: shniutan 时间: 2022-4-24 11:49
回复 9# qixiaobin0715
谢谢,之前发的是和实际不太一样,实际文本就是8楼的3个文件
作者: qixiaobin0715 时间: 2022-4-24 11:54
回复 10# shniutan
最好还是把文件上传,这样复制有时还是会与源文件有差距的。
作者: Batcher 时间: 2022-4-24 12:04
回复 10# shniutan
建议把真实文件打包上传到网盘
作者: shniutan 时间: 2022-4-24 13:48
回复 12# Batcher
谢谢,上传到哪个网盘?是通用网盘还是本网站自有网盘?
另外,请教个问题,问什么我回复帖子无法以图片或附件方式上传图片,图片小于50k,png格式的
作者: Batcher 时间: 2022-4-24 14:17
回复 13# shniutan
如果需要上传文件,可以用阿里云盘或百度网盘。
如果需要上传截图,可以找个图床,例如:
http://bbs.bathome.net/thread-60985-1-1.html
作者: shniutan 时间: 2022-4-24 15:29
回复 11# qixiaobin0715
感谢!文件已上传网盘。
链接:https://pan.baidu.com/s/1uAttTDidnZ1xVluD1oTh_w
提取码:nlg7
作者: shniutan 时间: 2022-4-24 15:29
回复 14# Batcher
谢谢指导
作者: qixiaobin0715 时间: 2022-4-24 15:40
回复 15# shniutan
写的有点乱,凑合用吧。bat文件存为ANSI编码:- @echo off
- setlocal enabledelayedexpansion
- for /f "tokens=7,12 delims=&= " %%i in (b.txt) do set #%%j=%%i
- (for /f "tokens=1* delims==" %%i in (a.txt) do echo,%%j)>Newfile.txt
- findstr /g:Newfile.txt c.txt>1.log
- (for /f "tokens=1* delims==" %%i in (1.log) do echo,%%j)>Newfile.txt
- for /f "tokens=1-5,7 delims=- " %%a in (Newfile.txt) do set /a "_%%a%%b%%c%%d%%e%%f+=1"
- (for /f "tokens=1-5,7 delims=- " %%a in (Newfile.txt) do (
- if not defined @%%a%%b%%c%%d%%e%%f (
- echo,%%f !#%%f! %%a-%%b-%%c-%%d-%%e !_%%a%%b%%c%%d%%e%%f!
- set /a Sum+=!_%%a%%b%%c%%d%%e%%f!
- )
- set @%%a%%b%%c%%d%%e%%f=true
- ))>1.log
- sort 1.log>Newfile.txt
- echo,Sum !Sum!>>Newfile.txt
- del 1.log
- pause
复制代码
作者: xczxczxcz 时间: 2022-4-24 19:46
本帖最后由 xczxczxcz 于 2022-4-24 19:48 编辑
给你一个好玩的,保存为 xxx.ps1, 右键执行- $a='.\a.txt';
- $b='.\b.txt';
- $c='.\c.txt';
-
- $hash=[Collections.Generic.HashSet[string]]::new();
- $infoID = @();
- (gc $a -ReadCount 0).foreach{
- $id = $_ -replace '^.*?=|\s+\d+$';
- if($hash.Add($id)){ $infoID+=$id }
- }
-
- $hash.Clear();
-
- $user = @();
- (gc $b -ReadCount 0).Foreach{
- $s = $_ -replace '^.*?=(.*?)&.*?-\s+((\d+\.){3}\d+).*$','$2,$1' -split ',';
- if($hash.Add($s[0]+$s[1])){
- $user += @{ $s[0].ToString() = $s[1]}
- }
- }
- $hash.Clear();
-
- $ipCount=@();
- (gc $c -ReadCount 0).Foreach{
- $s = $_ -replace '^.*?=(.*?)\s.*?-\s+((\d+\.){3}\d+).*$','$1,$2' -split ',';
- if($infoID.contains($s[0])){
- $ipCount+=[PsCustomObject]@{id=$s[0]; ip=$s[1]; name=$user."$($s[1])"; }
- }
- }
-
- ($ipcount|group name).Foreach{
- $ip =$_.Group[0].ip;
- $name = $_.Name;
- $_.group|group id|%{$ip + "`t" + $name + "`t`t" + $_.Name + "`t" + $_.Count}
- }
-
-
- &cmd /c set/p=全部完成 退出<nul`&pause>nul;exit
复制代码
作者: xczxczxcz 时间: 2022-4-24 19:47
这是结果:- 192.168.30.20 技术部_龙一a 7717F5D7-A409-4D48-9E87-1A92367C3593 1
- 192.168.30.20 技术部_龙一a C8FC9543-453E-44AB-9188-FC72963689D1 1
- 192.168.30.20 技术部_龙一a AAA31733-09CE-4015-A327-7C4BAC2B25EE 2
- 192.168.30.20 技术部_龙一a A0C8BDE0-6A46-4128-B566-5F40D0B83299 1
- 192.168.30.1 技术部_郭肥来a 7717F5D7-A409-4D48-9E87-1A92367C3593 2
- 192.168.30.1 技术部_郭肥来a 99D32186-121E-4FF7-AB54-0F816687A0A5 1
- 192.168.30.1 技术部_郭肥来a A0C8BDE0-6A46-4128-B566-5F40D0B83299 1
- 192.168.102.11 服务部_李四博a 7717F5D7-A409-4D48-9E87-1A92367C3593 1
- 192.168.102.11 服务部_李四博a 99D32186-121E-4FF7-AB54-0F816687A0A5 1
- 192.168.102.11 服务部_李四博a C8FC9543-453E-44AB-9188-FC72963689D1 1
- 192.168.102.11 服务部_李四博a AAA31733-09CE-4015-A327-7C4BAC2B25EE 1
- 192.168.29.10 人事部_肖逄文a 99D32186-121E-4FF7-AB54-0F816687A0A5 1
- 192.168.29.10 人事部_肖逄文a A0C8BDE0-6A46-4128-B566-5F40D0B83299 1
- 192.168.37.153 工程部_张三a 99D32186-121E-4FF7-AB54-0F816687A0A5 1
- 192.168.37.153 工程部_张三a C8FC9543-453E-44AB-9188-FC72963689D1 2
- 192.168.37.153 工程部_张三a A0C8BDE0-6A46-4128-B566-5F40D0B83299 1
- 192.168.81.18 投诉部_王欣零a AAA31733-09CE-4015-A327-7C4BAC2B25EE 1
复制代码
作者: shniutan 时间: 2022-5-5 08:40
回复 19# xczxczxcz
试了试,好像不行,用楼上的代码可以了,非常感谢
作者: shniutan 时间: 2022-5-5 08:40
回复 17# qixiaobin0715
用这个代码没问题,非常感谢
欢迎光临 批处理之家 (http://bbs.bathome.net/) |
Powered by Discuz! 7.2 |