勒色的Blog

In our work, we often encounter the following questions:
在我们的工作中，我经常遇到下面的问题：
I have two files: file1 and file2:
有两个文件：文件1和文件2：
1) How can I print out the lines that are only contained in file1?
1) 如何打印出只存在于文件1中的内容？
2) How can I print out the lines that are only contained in file2?
2) 如何打印出只存在于文件2中的内容？
3) How can I print out the lines that are contained both in file1 and file2?
3) 如何打印出文件1和文件2都有的内容？

There is a powerful shell command that can easily meet our needs, it is: comm.
这有一个很好的shell命令能够满足我们的需求，它就是comm。

When you meet the above questions, "comm" should be your first choice:-)
当你遇到上面的问题，“comm”应该是你第一选择:-)

comm [ -123 ]??file1??file2

comm will read file1 and file2 and generate three columns of output:
comm 将会读取文件1和文件2并且产生三列输出：
lines only in file1; lines only??in file2; and lines in both files.
只存在文件1中的行；只存在文件2中的行；两个文件都存在的行。
For detailed explanation, pls man comm.
更详细的解释，请参阅man comm。

Example:
例如：

bash-2.03$ cat file1
11111111
22222222
33333333
44444444
55555555
66666666
77777777
88888888
99999999
bash-2.03$ cat file2
00000000
22222222
44444444
66666666
88888888

1) suppress lines unique to FILE1
1) 过滤掉file1中的内容
bash-2.03$ comm -1 file1 file2
00000000
        22222222
        44444444
        66666666
        88888888

2) suppress lines unique to FILE2
2) 过滤掉file2中的内容
bash-2.03$ comm -2 file1 file2
11111111
        22222222
33333333
        44444444
55555555
        66666666
77777777
        88888888
99999999

3) suppress lines that appear in both files
3) 过滤掉file1和file2中都有的内容
bash-2.03$ comm -3 file1 file2
        00000000
11111111
33333333
55555555
77777777
99999999

4) Print out the lines that are only contained in file1?
4) 打印出只存在于文件1中的内容？
bash-2.03$ comm -23 file1 file2
11111111
33333333
55555555
77777777
99999999

5) Print out the lines that are only contained in file2?
5) 打印出只存在于文件2中的内容？
bash-2.03$ comm -13 file1 file2
00000000

6) Print out the lines that are contained both in file1 and file2?
6) 打印出文件1和文件2都有的内容？
bash-2.03$ comm -12 file1 file2
22222222
44444444
66666666
88888888

Besides the comm, we still have various ways to finish the above tasks.
除了comm，我们还有其他方法来完成这些任务。

4) Print out the lines that are only contained in file1?
4) 打印出只存在于文件1中的内容？
diff file1 file2 | grep "^<"|sed 's/^< //g'
for i in $(<file1); do (grep $i file2)||echo $i>>temp ; done;
cat temp

In comparison, comm is much easier to remember. :-)
相比之下，comm更加便于记忆。

转自：http://hi.baidu.com/will_hu/blog/item/4b05fedf0276fd5fcdbf1a6d.html

#!/usr/bin/ksh
#
# SCRIPT: 12_ways_to_parse.ksh.ksh
#
#
# REV: 1.2.A
#
# PURPOSE: This script shows the different ways of reading
#       a file line by line. Again there is not just one way
#       to read a file line by line and some are faster than
#       others and some are more intuitive than others.
#
# REV LIST:
#
#       03/15/2002 - Randy Michael
#       Set each of the while loops up as functions and the timing
#       of each function to see which one is the fastest.
#
#######################################################################
#
#       NOTE: To output the timing to a file use the following syntax:
#
#          12_ways_to_parse.ksh file_to_process > output_file_name 2>&1
#
#       The actaul timing data is sent to standard error, file
#       descriptor (2), and the function name header is sent
#       to standard output, file descriptor (1).
#
#######################################################################
#
# set -n # Uncomment to check command syntax without any execution
# set -x # Uncomment to debug this script
#

FILENAME="$1"
TIMEFILE="/tmp/loopfile.out"
>$TIMEFILE
THIS_SCRIPT=$(basename $0)

######################################
function usage
{
echo "\nUSAGE: $THIS_SCRIPT file_to_process\n"
echo "OR - To send the output to a file use: "
echo "\n$THIS_SCRIPT file_to_process > output_file_name 2>&1 \n"
exit 1
}
######################################
function while_read_LINE
{
cat $FILENAME | while read LINE
do
        echo "$LINE"
        :
done
}
######################################
function while_read_LINE_bottom
{
while read LINE
do
        echo "$LINE"
        :

done < $FILENAME
}
######################################
function while_line_LINE_bottom
{
while line LINE
do
        echo $LINE
        :
done < $FILENAME
}
######################################
function cat_while_LINE_line
{
cat $FILENAME | while LINE=`line`
do
        echo "$LINE"
        :
done
}
######################################
function while_line_LINE
{
cat $FILENAME | while line LINE
do
        echo "$LINE"
        :
done
}
######################################
function while_LINE_line_bottom
{
while LINE=`line`
do
        echo "$LINE"
        :

done < $FILENAME
}
######################################
function while_LINE_line_cmdsub2
{
cat $FILENAME | while LINE=$(line)
do
        echo "$LINE"
        :
done
}
######################################
function while_LINE_line_bottom_cmdsub2
{
while LINE=$(line)
do
        echo "$LINE"
        :

done < $FILENAME
}
######################################
function while_read_LINE_FD
{
exec 3<&0
exec 0< $FILENAME
while read LINE
do
        echo "$LINE"
        :
done
exec 0<&3
}
######################################
function while_LINE_line_FD
{
exec 3<&0
exec 0< $FILENAME
while LINE=`line`
do
        echo "$LINE"
        :
done
exec 0<&3
}
######################################
function while_LINE_line_cmdsub2_FD
{
exec 3<&0
exec 0< $FILENAME
while LINE=$(line)
do
        print "$LINE"
        :
done
exec 0<&3
}
######################################
function while_line_LINE_FD
{
exec 3<&0
exec 0< $FILENAME

while line LINE
do
        echo "$LINE"
        :
done

exec 0<&3
}
######################################
########### START OF MAIN ############
######################################

# Test the Input

# Looking for exactly one parameter
(( $# == 1 )) || usage

# Does the file exist as a regular file?
[[ -f $1 ]] || usage

echo "\nStarting File Processing of each Method\n"

echo "Method 1:"
echo "\nfunction while_read_LINE\n" >> $TIMEFILE
echo "function while_read_LINE"
time while_read_LINE >> $TIMEFILE
echo "\nMethod 2:"
echo "\nfunction while_read_LINE_bottom\n" >> $TIMEFILE
echo "function while_read_LINE_bottom"
time while_read_LINE_bottom >> $TIMEFILE
echo "\nMethod 3:"
echo "\nfunction while_line_LINE_bottom\n" >> $TIMEFILE
echo "function while_line_LINE_bottom"
time while_line_LINE_bottom >> $TIMEFILE
echo "\nMethod 4:"
echo "\nfunction cat_while_LINE_line\n" >> $TIMEFILE
echo "function cat_while_LINE_line"
time cat_while_LINE_line >> $TIMEFILE
echo "\nMethod 5:"
echo "\nfunction while_line_LINE\n" >> $TIMEFILE
echo "function while_line_LINE"
time while_line_LINE >> $TIMEFILE
echo "\nMethod 6:"
echo "\nfunction while_LINE_line_bottom\n" >> $TIMEFILE
echo "function while_LINE_line_bottom"
time while_LINE_line_bottom >> $TIMEFILE
echo "\nMethod 7:"
echo "\nfunction while_LINE_line_cmdsub2\n" >> $TIMEFILE
echo "function while_LINE_line_cmdsub2"
time while_LINE_line_cmdsub2 >> $TIMEFILE
echo "\nMethod 8:"
echo "\nfunction while_LINE_line_bottom_cmdsub2\n" >> $TIMEFILE
echo "function while_LINE_line_bottom_cmdsub2"
time while_LINE_line_bottom_cmdsub2 >> $TIMEFILE
echo "\nMethod 9:"
echo "\nfunction while_read_LINE_FD\n" >> $TIMEFILE
echo "function while_read_LINE_FD"
time while_read_LINE_FD >> $TIMEFILE
echo "\nMethod 10:"
echo "\nfunction while_LINE_line_FD\n" >> $TIMEFILE
echo "function while_LINE_line_FD"
time while_LINE_line_FD >> $TIMEFILE
echo "\nMethod 11:"
echo "\nfunction while_LINE_line_cmdsub2_FD\n" >> $TIMEFILE
echo "function while_LINE_line_cmdsub2_FD"
time while_LINE_line_cmdsub2_FD >> $TIMEFILE
echo "\nMethod 12:"
echo "\nfunction while_line_LINE_FD\n" >> $TIMEFILE
echo "function while_line_LINE_FD"
time while_line_LINE_FD >> $TIMEFILE