newswan

回复 28# qixiaobin0715


vbs 不会，但是 powershell 效率低，是很无语的。ms 一向是这样
批处理+第三方 有时候也麻烦。

newswan

gnu命令： sed sort

@echo off
setlocal ENABLEDELAYEDEXPANSION

set path=%pathgnu%;%path%
echo %path%

set sour=新建文件夹
set dest=bbb1
set exclude=QQ

del /q "%dest%\*.*"

echo %~nx0 %time% 1 >>time.txt

for /f "usebackq tokens=* delims=" %%a in (`dir /s /b "%sour%\*.txt"`) do (
    echo "%%~fa"
    type "%%~fa" | sed -r -e "/\S+\s+\S+\s+\S+/^!d" -e "/%exclude%/d" -e "s/SZ/1/" -e "s/SH/0/" -e "s/\s+/^|/g"  >> "%dest%\%%~na.tmp"
)

echo,
echo %~nx0 %time% 2 >>time.txt
echo,

for /f "usebackq tokens=* delims=" %%a in (`dir /s /b "%dest%\*.tmp"`) do (
    echo "%%~fa"
    type "%%~fa" | sort.exe -u >>"%%~dpna.txt"
)

echo %~nx0 %time% 3 >>time.txt
echo,
del %dest%\*.tmp
复制代码

newswan

gnu 命令 sed awk

@echo off
setlocal ENABLEDELAYEDEXPANSION

set path=%pathgnu%;%path%
echo %path%

set sour=新建文件夹
set dest=bbb2
set exclude=QQ

del /q "%dest%\*.*"

echo %~nx0 %time% 1 >>time.txt

for /f "usebackq tokens=* delims=" %%a in (`dir /s /b "%sour%\*.txt"`) do (
    echo "%%~fa"
    type "%%~fa" | sed -r -e "/\S+\s+\S+\s+\S+/^!d" -e "/%exclude%/d" -e "s/SZ/1/" -e "s/SH/0/" -e "s/\s+/^|/g"  >> "%dest%\%%~na.tmp"
)

echo,
echo %~nx0 %time% 2 >>time.txt
echo,

for /f "usebackq tokens=* delims=" %%a in (`dir /s /b "%dest%\*.tmp"`) do (
    echo "%%~fa"
    type "%%~fa" | awk " { arr[$0]++ ; if ( arr[$0] == 1 ) { print $0 } } "  >>"%%~dpna.txt"
)

echo %~nx0 %time% 3 >>time.txt
echo,
del %dest%\*.tmp
复制代码

newswan

效率 比较
sed + sort
1.bat 16:01:09.17 1
1.bat 16:02:06.45 2
1.bat 16:02:21.43 3
sed + awk
2.bat 16:03:13.41 1
2.bat 16:04:11.68 2
2.bat 16:04:26.42 3

两种办法 效率相同，但是 ps 就差很多了。

newswan

回复 39# idwma


谢谢指点
net 没学过。。。
合并效率一样了，去重，还是很慢

$sour = "D:\share\tech\New folder\新建文件夹"
$dest = "D:\share\tech\New folder\ccc2"
$exclude = "QQ"

Remove-Item $dest\*.*

( $MyInvocation.MyCommand.Name + "  1  " + (get-date -Format "HH:mm:ss.ff").tostring() ) | out-file -Encoding ascii -append time.txt

Get-ChildItem -path $sour *.txt -Recurse | foreach-object {
    $_.fullname
    $filename = Join-Path -path $dest -ChildPath ($_.basename + ".tmp")
    $a=( [IO.File]::ReadAllLines($_.fullname) )
    $a = $a -match "\S+\s+\S+\s+\S+" -notmatch $exclude -replace "SZ","1" -replace "SH","0" -replace "\s+","|"
    [IO.File]::AppendAllLines([string]$filename , [string[]]$a)
}

( $MyInvocation.MyCommand.Name + "  2  " + (get-date -Format "HH:mm:ss.ff").tostring() ) | out-file -Encoding ascii -append time.txt

Get-ChildItem -path $dest *.tmp | foreach-object {
    $_.fullname
    $filename = Join-Path -path $dest -ChildPath ($_.basename + ".txt")
    $ht = @{}
    $a = ( [IO.File]::ReadAllLines($_.fullname) )
    $a = $a | foreach-object {
        if ( -not ( $ht.ContainsKey($_) ) )
        {
            $_
            $ht.add($_,"1")
        }
    }
    $a | out-file -Encoding utf8 $filename
}

( $MyInvocation.MyCommand.Name + "  3  " + (get-date -Format "HH:mm:ss.ff").tostring() ) | out-file -Encoding ascii -append time.txt
复制代码

newswan

回复 42# idwma


谢谢，这个用时是 unix 工具的 6 倍，比前面的要好多了

newswan

一段时间不用，很多都不熟悉了啊

newswan

相当于

 $null = 
复制代码

newswan

同名文件分组，流式处理
需要 sed awk sort

@echo off
setlocal ENABLEDELAYEDEXPANSION

set sour=新建文件夹
set dest=%~n0
set exclude=QQ

set "_cmd_=dir /s /b %sour%\*.txt | sed -r -e "s/^.+[\]//" | sort -u"
set _cmdsed_=-r -e "/%exclude%/d" -e "/^\S+\s+\S+\s+\S+$/^!d" -e "s/SZ/1/" -e "s/SH/0/" -e "s/\s+/^|/g"
set _cmdawk_=" { arr[$0]++ ; if ( arr[$0] == 1 ) { print $0 } } "
for /f "usebackq tokens=* delims=" %%a in (`cmd /c "%_cmd_%"`) do (
    echo %%a
    (
        for /f "usebackq tokens=* delims=" %%b in (`dir /s /b "%sour%\%%a"`) do (
            type "%%~fb"
        ) | sed -r -e $a\
    ) | sed !_cmdsed_! | awk !_cmdawk_! > "%dest%\%%~nxa"
)
复制代码

ps: linux 的 sort 不是 windows自带的

newswan

回复 52# xczxczxcz


可以在vbs里加个计时器，完成后显示时长
这样时间比较准确

newswan

@echo off
setlocal ENABLEDELAYEDEXPANSION
chcp 936

set path=%pathgnu%;%path%

set sour=新建文件夹
set dest=%~n0
set exclude=QQ

if exist %dest% rd /s /q %dest%
if not exist %dest% mkdir %dest%

call :C_timer

set "_cmd_=dir /s /b %sour%\*.txt | sed -r -e "s/^.+[\]//" | sort -u"
set _cmdsed_=-r -e "/%exclude%/d" -e "/^\S+\s+\S+\s+\S+$/^!d" -e "s/SH/0/" -e "s/SZ/1/" -e "s/\s+/^|/g"
set _cmdawk_=" { arr[$0]++ ; if ( arr[$0] == 1 ) { print $0 } } "
for /f "usebackq tokens=* delims=" %%a in (`cmd /c "%_cmd_%"`) do (
    echo %%a
    (
        for /f "usebackq tokens=* delims=" %%b in (`dir /s /b "%sour%\%%a"`) do (
            type "%%~fb"
        ) | sed -r -e $a\
    ) | sed !_cmdsed_! | awk !_cmdawk_! > "%dest%\%%~nxa"
)

call :C_timer sum

exit/b

rem del %dest%\*.tmp


:C_timer
if not defined _ti_ set _ti_=-1
set/a _ti_+=1
set _tb_=%_te_%
set _te_=%time%
if %_ti_% EQU 0 (
    set _tb_=%_te_%
)
set/a _tdiff_=(9%_te_:~0,2%-9%_tb_:~0,2%)*360000+(9%_te_:~3,2%-9%_tb_:~3,2%)*6000+(9%_te_:~6,2%%_te_:~9,2%-9%_tb_:~6,2%%_tb_:~9,2%)
set/a _tdiff_=%_tdiff_:-=8640000-%
set/a _tdiffSum_+=%_tdiff_%
if "%_ti_%" == "0" (
    (echo,) >>time.txt
    (echo %~nx0  %_ti_%  %_te_%) >>time.txt
) else (
    (echo %~nx0  %_ti_%  %_te_%  %_tdiff_:~0,-2%.%_tdiff_:~-2%) >>time.txt
)
if %1.==sum. (
    (echo %~nx0                  %_tdiffSum_:~0,-2%.%_tdiffSum_:~-2%) >>time.txt
)
goto :eof
复制代码

newswan

回复 55# xczxczxcz


我的意思是加到vbs代码里，运行完毕显示用时
MsgBox "Done 用时 xx.xx "

newswan

回复 59# xczxczxcz


😓，搞错，是给 why 回复的。

没有禁用表情，怎么表情发不出

newswan

回复 50# WHY


可以加个计时功能，运行完成，显示用时