[Question] Download Attachments or Files from Websites with Java and HttpClient

[Psycho_Coder]

Hello Everyone,

I am having a little problem while downloading a file from a site.

Aim

Download a File (named MyContacts.xls) from a site after logging into the site.

My Approach

1. Login to the Site and store the Session IDs and cookies. ConnectionManager is a class which manages the login process in my code and stores the cookies or session id.

2. http://site.com/ExportContacts?Token=<Some-Token-ID>, this is the url which when I visit after logging into the site manually the file MyContactc.xls is downloaded. I want to download this using code.

3. I sent a POST request and set the headers as required. I observed the headers by inspecting the webpage and monitoring the requests sent under the network tab in Chrome.





These are the Request and Response Headers. In the second image the form data I am primarily interested in that data which is downloaded as xls file. But I am unable to get the data as well. Whatever I am more concerned with downloading the file using Code.

Libraries Used

Apache HttpClient

Below is the main snippet where I am trying to download the file.

Code: (java) [Select]

try {
            ConnectionManager conMan = new ConnectionManager("Some Username", "Some Password");
            conMan.login();

            HttpClient client = HttpClientBuilder.create().build();

            HttpPost post = new HttpPost("http://site.com/ExportContacts");

            post.setHeader("User-Agent", Utils.USERAGENT.getValue());
            post.setHeader("Content-Type", "application/msexcel");
            post.setHeader("Content-Disposition", "attachment; filename=MyContacts.xls");

            List<NameValuePair> paramList = new ArrayList<>();
            paramList.add(new BasicNameValuePair("Token", conMan.getToken()));

            post.setEntity(new UrlEncodedFormEntity(paramList));

            HttpResponse resp = client.execute(post);
            HttpEntity entity = resp.getEntity();

            InputStream inStream = entity.getContent();
            FileOutputStream outputStream = new FileOutputStream("MyContacts.xls");

            int bytesRead = -1;
            byte[] buffer = new byte[512];
            while ((bytesRead = inStream.read(buffer)) != -1) {
                outputStream.write(buffer, 0, bytesRead);
            }

            outputStream.close();
            inStream.close();
        } catch (IOException ex) {
               log4jLogger.error("IOException Occured", ex);
        }


I hope to get the solution, alternative methods to get this thing done. Please note that I did came across a few solutions on StackOverflow and a few other places but they aren't helpful I have tried them already. Similer examples on site like http://www.mkyong.com/java/how-to-download-file-from-website-java-jsp/, I have tried but without success.


Please Note that I cannot give the code for other files since they are independent of this module. Moreover its a bigger project, so I can't give out any code as of now.



Thank you,
Sincerely,
Psycho_Coder.