1. 程式人生 > >java對ftp操作遇到的一系列的坑

java對ftp操作遇到的一系列的坑

首先讓我先講一下最近專案中遇到的伺服器主要使用的ftp伺服器構建工具 (IIS 、ser-u、filezilla)

IIS(摘錄自百度百科)

  • IIS 是widow中IIS是一個World Wide Web server。Gopher server和FTP server全部包容在裡面。 IIS意味著你能釋出網頁,並且有ASP(Active Server Pages)、JAVA、VBscript產生頁面,有著一些擴充套件功能。IIS支援一些有趣的東西,像有編輯環境的介面(FRONTPAGE)、有全文檢索功能的(INDEX SERVER)、有多媒體功能的(NET SHOW) 其次,IIS是隨Windows NT Server 4.0一起提供的檔案和應用程式伺服器,是在Windows NT Server上建立Internet伺服器的基本元件。它與Windows NT Server完全整合,允許使用Windows NT Server內建的安全性以及NTFS檔案系統建立強大靈活的Internet/Intranet站點。IIS(Internet Information Server,網際網路資訊服務)是一種Web(網頁)服務元件,其中包括Web伺服器、FTP伺服器、NNTP伺服器和SMTP伺服器,分別用於網頁瀏覽、檔案傳輸、新聞服務和郵件傳送等方面,它使得在網路(包括網際網路和區域網)上釋出資訊成了一件很容易的事。

ser-u(摘錄自百度百科)

  • Serv-U™ - Windows平臺的FTP伺服器軟體Serv-U 是目前眾多的FTP 伺服器軟體之一。通過使用Serv-U,使用者能夠將任何一臺PC 設定成一個FTP 伺服器,這樣,使用者或其他使用者就能夠使用FTP 協議,通過在同一網路上的任何一臺PC與FTP 伺服器連線,進行檔案或目錄的複製,移動,建立,和刪除等。這裡提到的FTP 協議是專門被用來規定計算機之間進行檔案傳輸的標準和規則,正是因為有了像FTP 這樣的專門協議,才使得人們能夠通過不同型別的計算機,使用不同型別的作業系統,對不同型別的檔案進行相互傳遞。

filezilla(摘錄百度百科)

  • FileZilla是一個免費開源的FTP軟體,分為客戶端版本和伺服器版本,具備所有的FTP軟體功能。可控性、有條理的介面和管理多站點的簡化方式使得Filezilla客戶端版成為一個方便高效的FTP客戶端工具,而FileZilla Server則是一個小巧並且可靠的支援FTP&SFTP的FTP伺服器軟體。

java中操作ftp api-ftpclient

開始記錄在使用這兩個ftp構建工具遇到的一些問題(win7 平臺)

  • ftp 某些方法 如listFile 等方法無效,卡死
    這個問題是兩個工具中我都遇到了 ,這個問題是關於主動模和被動模式有關,這裡我貼出我在看了相關的博文的總結:
    1. 主動模式
      是可以這樣想服務端的埠確定 比如命令埠21埠,還有資料埠20埠 主動連結放在客戶端,客戶端會開啟一個大於1023埠去連結伺服器21埠,還有開啟N+1埠去連結伺服器的資料埠,並監聽這兩個埠
      而客戶端又要向服務端開放埠 ,這個是客戶端開發由外向內的埠連結(在資料埠上面),,,所以一般會被防火牆攔截 (埠隨機)
      所以一般會出現可以連線上,但是訪問資料會出現異常的現象,就比如今天我用eclipse 的時候出現的問題,呼叫listFile 會出現卡死在建立socket上面了
    2. 被動模式
      被動方式與主動方式不同的是,先是一樣的21埠連結,但是連結成功後,伺服器端先開啟一個數據埠,告知客戶端通過本地的埠來連線自己的這個開放的埠,一般的ftp伺服器都允許客戶端通過高階口位連結自己。

卡死,由於ftpclient 操作預設是是通過主動模式 ,可以設定為被動模式(但是自己當時使用eclipse 會出現這個問題 ,使用idea卻沒有問題 ,跟蹤程式碼是在卡死在了建立socket上面)

            ftpClient.enterLocalPassiveMode();
  • ftp通過ser-u構建,出現的問題
    ftpclient listFile方法無法返回正確的資料,一般返回時null ,使用listNames 返回的也是隻有檔名,這個是ftpclient 工具包的一個bug,這個有專門的大神分析了原始碼,並給出了多種的解決辦法,比如我使用了下面這種(分析太多,,有機會仔細看下)
ftpClient.configure(new FTPClientConfig("com.zznode.tnms.ra.c11n.nj.resource.ftp.UnixFTPEntryParser"));

奇怪的是使用IIS 構建的伺服器並不會出現這個問題

  • 使用ser-u構建 ftp伺服器獲取檔案列表,會獲取當前目錄和上一級目錄
    估計是為了相容linux 和window不同的平臺吧 ,,不是吧許可權模型也改成了和linux一樣的rwx嘛,但是IIS 不會反悔上一級和當前的目錄,所以如果是使用ser-u 構建的ftp,那麼在進行ftp檔案無限層級檢索的時候要記得過濾
.   ..

這兩個目錄 ,不然會死迴圈~~~

  • filezilla 出現listfile為空的情況(好像是之前使用ser-u也出現類似的情況)
    網上說是解決ftp支援系統中文環境,建立下面兩個類檔案:

    package com.xpp.util.filezillapatch;
    import java.text.ParseException;
    import java.text.ParsePosition;
    import java.text.SimpleDateFormat;
    import java.util.Calendar;
    import java.util.Date;
    import org.apache.commons.net.ftp.parser.FTPTimestampParserImpl;
    public class FTPTimestampParserImplExZH extends FTPTimestampParserImpl {
        private SimpleDateFormat defaultDateFormat = new SimpleDateFormat("mm d hh:mm");
        private SimpleDateFormat recentDateFormat = new SimpleDateFormat("yyyy mm d");
        private String formatDate_Zh2En(String timeStrZh) {
            if (timeStrZh == null) {
                return "";
            }
            int len = timeStrZh.length();
            StringBuffer sb = new StringBuffer(len);
            char ch = ' ';
            for (int i = 0; i < len; i++) {
                ch = timeStrZh.charAt(i);
                if ((ch >= '0' && ch <= '9') || ch == ' ' || ch == ':') {
                    sb.append(ch);
                }
            }
            return sb.toString();
        }
        /**
         * Implements the one {@link FTPTimestampParser#parseTimestamp(String) method} in the {@link FTPTimestampParser
         * FTPTimestampParser} interface according to this algorithm: If the recentDateFormat member has been defined, try
         * to parse the supplied string with that. If that parse fails, or if the recentDateFormat member has not been
         * defined, attempt to parse with the defaultDateFormat member. If that fails, throw a ParseException.
         *
         * @see org.apache.commons.net.ftp.parser.FTPTimestampParser#parseTimestamp(String)
         */
        public Calendar parseTimestamp(String timestampStr) throws ParseException {
            timestampStr = formatDate_Zh2En(timestampStr);
            Calendar now = Calendar.getInstance();
            now.setTimeZone(this.getServerTimeZone());
    
            Calendar working = Calendar.getInstance();
            working.setTimeZone(this.getServerTimeZone());
            ParsePosition pp = new ParsePosition(0);
    
            Date parsed = null;
            if (this.recentDateFormat != null) {
                parsed = recentDateFormat.parse(timestampStr, pp);
            }
            if (parsed != null && pp.getIndex() == timestampStr.length()) {
                working.setTime(parsed);
                working.set(Calendar.YEAR, now.get(Calendar.YEAR));
                if (working.after(now)) {
                    working.add(Calendar.YEAR, -1);
                }
            } else {
                pp = new ParsePosition(0);
                parsed = defaultDateFormat.parse(timestampStr, pp);
                // note, length checks are mandatory for us since
                // SimpleDateFormat methods will succeed if less than
                // full string is matched. They will also accept,
                // despite "leniency" setting, a two-digit number as
                // a valid year (e.g. 22:04 will parse as 22 A.D.)
                // so could mistakenly confuse an hour with a year,
                // if we don't insist on full length parsing.
                if (parsed != null && pp.getIndex() == timestampStr.length()) {
                    working.setTime(parsed);
                } else {
                    throw new ParseException("Timestamp could not be parsed with older or recent DateFormat", pp.getIndex());
                }
            }
            return working;
        }
    }
    

第二個檔案

    package com.xpp.util.filezillapatch;
    /*
     * Copyright 2001-2005 The Apache Software Foundation
     *
     * Licensed under the Apache License, Version 2.0 (the "License");
     * you may not use this file except in compliance with the License.
     * You may obtain a copy of the License at
     *
     *     http://www.apache.org/licenses/LICENSE-2.0
     *
     * Unless required by applicable law or agreed to in writing, software
     * distributed under the License is distributed on an "AS IS" BASIS,
     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     * See the License for the specific language governing permissions and
     * limitations under the License.
     */

    import java.text.ParseException;
    import java.util.Calendar;

    import org.apache.commons.net.ftp.FTPClientConfig;
    import org.apache.commons.net.ftp.FTPFile;
    import org.apache.commons.net.ftp.parser.ConfigurableFTPFileEntryParserImpl;
    import org.apache.log4j.Logger;

    /**
     * 注:common-net-1.4.1.jar原始碼,修改對於日期中文格式的支援,從而解決FTPClient.listFiles()返回為空問題
     * Implementation FTPFileEntryParser and FTPFileListParser for standard
     * Unix Systems.
     *
     * This class is based on the logic of Daniel Savarese's
     * DefaultFTPListParser, but adapted to use regular expressions and to fit the
     * new FTPFileEntryParser interface.
     * @version $Id: UnixFTPEntryParser.java 161712 2005-04-18 02:57:04Z scohen $
     * @see org.apache.commons.net.ftp.FTPFileEntryParser FTPFileEntryParser (for usage instructions)
     */
    public class UnixFTPEntryParser extends ConfigurableFTPFileEntryParserImpl
    {
        private static Logger logger = Logger.getLogger(UnixFTPEntryParser.class);
        /**
         * months abbreviations looked for by this parser.  Also used
         * to determine which month is matched by the parser
         */
        private static final String DEFAULT_MONTHS =
                "(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)";

        static final String DEFAULT_DATE_FORMAT
                = "MMM d yyyy"; //Nov 9 2001

        static final String DEFAULT_RECENT_DATE_FORMAT
                = "MMM d HH:mm"; //Nov 9 20:06

        static final String NUMERIC_DATE_FORMAT
                = "yyyy-MM-dd HH:mm"; //2001-11-09 20:06

        /**
         * Some Linux distributions are now shipping an FTP server which formats
         * file listing dates in an all-numeric format:
         * <code>"yyyy-MM-dd HH:mm</code>.
         * This is a very welcome development,  and hopefully it will soon become
         * the standard.  However, since it is so new, for now, and possibly
         * forever, we merely accomodate it, but do not make it the default.
         * <p>
         * For now end users may specify this format only via
         * <code>UnixFTPEntryParser(FTPClientConfig)</code>.
         * Steve Cohen - 2005-04-17
         */
        public static final FTPClientConfig NUMERIC_DATE_CONFIG =
                new FTPClientConfig(
                        FTPClientConfig.SYST_UNIX,
                        NUMERIC_DATE_FORMAT,
                        null, null, null, null);

        /**
         * this is the regular expression used by this parser.
         *
         * Permissions:
         *    r   the file is readable
         *    w   the file is writable
         *    x   the file is executable
         *    -   the indicated permission is not granted
         *    L   mandatory locking occurs during access (the set-group-ID bit is
         *        on and the group execution bit is off)
         *    s   the set-user-ID or set-group-ID bit is on, and the corresponding
         *        user or group execution bit is also on
         *    S   undefined bit-state (the set-user-ID bit is on and the user
         *        execution bit is off)
         *    t   the 1000 (octal) bit, or sticky bit, is on [see chmod(1)], and
         *        execution is on
         *    T   the 1000 bit is turned on, and execution is off (undefined bit-
         *        state)
         */
        private static final String REGEX =
                "([bcdlfmpSs-])"
                        +"(((r|-)(w|-)([xsStTL-]))((r|-)(w|-)([xsStTL-]))((r|-)(w|-)([xsStTL-])))\\+?\\s+"
                        + "(\\d+)\\s+"
                        + "(\\S+)\\s+"
                        + "(?:(\\S+)\\s+)?"
                        + "(\\d+)\\s+"

            /*
              numeric or standard format date
            */
                        //問題出在此處,這個匹配只匹配2中形式:
                        //(1)2008-08-03
                        //(2)Jan  9或4月 26
                        //而出錯的hp機器下的顯示為 8月20日(沒有空格分開)
                        //故無法匹配而報錯
                        //將下面字串改為:
                        + "((?:\\d+[-/]\\d+[-/]\\d+)|(?:\\S+\\s+\\S+)|(?:\\S+))\\s+"
                        //+ "((?:\\d+[-/]\\d+[-/]\\d+)|(?:\\S+\\s+\\S+))\\s+"

            /*
               year (for non-recent standard format)
               or time (for numeric or recent standard format
            */
                        + "(\\d+(?::\\d+)?)\\s+"

                        + "(\\S*)(\\s*.*)";


        /**
         * The default constructor for a UnixFTPEntryParser object.
         *
         * @exception IllegalArgumentException
         * Thrown if the regular expression is unparseable.  Should not be seen
         * under normal conditions.  It it is seen, this is a sign that
         * <code>REGEX</code> is  not a valid regular expression.
         */
        public UnixFTPEntryParser()
        {
            this(null);
        }

        /**
         * This constructor allows the creation of a UnixFTPEntryParser object with
         * something other than the default configuration.
         *
         * @param config The {@link FTPClientConfig configuration} object used to
         * configure this parser.
         * @exception IllegalArgumentException
         * Thrown if the regular expression is unparseable.  Should not be seen
         * under normal conditions.  It it is seen, this is a sign that
         * <code>REGEX</code> is  not a valid regular expression.
         * @since 1.4
         */
        public UnixFTPEntryParser(FTPClientConfig config)
        {
            super(REGEX);
            configure(config);
        }


        /**
         * Parses a line of a unix (standard) FTP server file listing and converts
         * it into a usable format in the form of an <code> FTPFile </code>
         * instance.  If the file listing line doesn't describe a file,
         * <code> null </code> is returned, otherwise a <code> FTPFile </code>
         * instance representing the files in the directory is returned.
         * <p>
         * @param entry A line of text from the file listing
         * @return An FTPFile instance corresponding to the supplied entry
         */
        public FTPFile parseFTPEntry(String entry) {
            FTPFile file = new FTPFile();
            file.setRawListing(entry);
            int type;
            boolean isDevice = false;

            if (matches(entry))
            {
                String typeStr = group(1);
                String hardLinkCount = group(15);
                String usr = group(16);
                String grp = group(17);
                String filesize = group(18);
                String datestr = group(19) + " " + group(20);
                String name = group(21);
                String endtoken = group(22);

                try
                {
                    //file.setTimestamp(super.parseTimestamp(datestr));
                    FTPTimestampParserImplExZH Zh2En = new FTPTimestampParserImplExZH();
                    file.setTimestamp(Zh2En.parseTimestamp(datestr));
                }
                catch (ParseException e)
                {
                    //logger.error(e, e);
                    //return null;  // this is a parsing failure too.
                    //logger.info(entry+":修改日期重置為當前時間");
                    file.setTimestamp(Calendar.getInstance());
                }


                // bcdlfmpSs-
                switch (typeStr.charAt(0))
                {
                    case 'd':
                        type = FTPFile.DIRECTORY_TYPE;
                        break;
                    case 'l':
                        type = FTPFile.SYMBOLIC_LINK_TYPE;
                        break;
                    case 'b':
                    case 'c':
                        isDevice = true;
                        // break; - fall through
                    case 'f':
                    case '-':
                        type = FTPFile.FILE_TYPE;
                        break;
                    default:
                        type = FTPFile.UNKNOWN_TYPE;
                }

                file.setType(type);

                int g = 4;
                for (int access = 0; access < 3; access++, g += 4)
                {
                    // Use != '-' to avoid having to check for suid and sticky bits
                    file.setPermission(access, FTPFile.READ_PERMISSION,
                            (!group(g).equals("-")));
                    file.setPermission(access, FTPFile.WRITE_PERMISSION,
                            (!group(g + 1).equals("-")));

                    String execPerm = group(g + 2);
                    if (!execPerm.equals("-") && !Character.isUpperCase(execPerm.charAt(0)))
                    {
                        file.setPermission(access, FTPFile.EXECUTE_PERMISSION, true);
                    }
                    else
                    {
                        file.setPermission(access, FTPFile.EXECUTE_PERMISSION, false);
                    }
                }

                if (!isDevice)
                {
                    try
                    {
                        file.setHardLinkCount(Integer.parseInt(hardLinkCount));
                    }
                    catch (NumberFormatException e)
                    {
                        // intentionally do nothing
                    }
                }

                file.setUser(usr);
                file.setGroup(grp);

                try
                {
                    file.setSize(Long.parseLong(filesize));
                }
                catch (NumberFormatException e)
                {
                    // intentionally do nothing
                }

                if (null == endtoken)
                {
                    file.setName(name);
                }
                else
                {
                    // oddball cases like symbolic links, file names
                    // with spaces in them.
                    name += endtoken;
                    if (type == FTPFile.SYMBOLIC_LINK_TYPE)
                    {

                        int end = name.indexOf(" -> ");
                        // Give up if no link indicator is present
                        if (end == -1)
                        {
                            file.setName(name);
                        }
                        else
                        {
                            file.setName(name.substring(0, end));
                            file.setLink(name.substring(end + 4));
                        }

                    }
                    else
                    {
                        file.setName(name);
                    }
                }
                return file;
            } else {
                logger.info("matches(entry) failure:"+entry);
            }
            return null;
        }

        /**
         * Defines a default configuration to be used when this class is
         * instantiated without a {@link  FTPClientConfig  FTPClientConfig}
         * parameter being specified.
         * @return the default configuration for this parser.
         */
        protected FTPClientConfig getDefaultConfiguration() {
            return new FTPClientConfig(
                    FTPClientConfig.SYST_UNIX,
                    DEFAULT_DATE_FORMAT,
                    DEFAULT_RECENT_DATE_FORMAT,
                    null, null, null);
        }
    }

然後在你連線ftp的地方加上

 ftp.configure(new FTPClientConfig("com.xpp.util.filezillapatch.UnixFTPEntryParser"));

這樣就能返回正常的資料了