Post-Processing of data from Point HTC

This example demonstrates how to use PanPython SDK handling the raw data from HTC point calculation and perform statistical analysis of these data. In this example, raw data are commingled in a single merged CSV file (merged.csv), which is then visualized through a Jupyter notebook.

Point HTC is discussed in Parallel High-Throughput Point Calculation (HTC).

About point HTC

Point HTC allows a user to perform calculations for numerous compositions and/or temperatures in the user-defined state space by a simple setting. Alloy compositions and/or temperatures that satisfy user-defined criteria can then be discovered through data mining of the simulated results.

A step-by-step guide

Note

A video guide will be available very soon.

  1. Open the demo folder

    /Solution_Demos/panhtc_points_post_process/” from PyCharm IDE. See Installation to setup the IDE environment.

  2. Build a task script

    An example script panhtc_points_post_process.py has already been include. Following the steps the create your script:

    Import packages
    from panpython.sdk.stat.htc_result_post_processing import PandatHTCResultProcessor
    import os
    
    Setup working folders and input files:
    dir_name = os.path.abspath(__file__)
    file_name = os.path.basename(__file__)
    task_path = dir_name[:-len(file_name)]
    
    htc_result_parent_path = 'source/default'
    
    Setup table name of each point calculation:
    file_name = 'Default.table'
    
    Setup output path and output file name ‘merged.csv’
    dump_path = os.path.join(task_path, 'output')
    file_output = 'merged.csv'
    
    Run post-processing of point HTC:
    if __name__ == "__main__":
        m_processor = PandatHTCResultProcessor(parent_path=htc_result_parent_path,
                                               file_name=os.path.join('Table', file_name),
                                               cal_type='point')
        m_processor.save_to_csv(output_path=dump_path, output_file=file_output)
    
  3. Run the post-processing script

    After the post-processing is finished, a generated csv file (merged.csv) is ready to be used. See The inputs and outputs of this post-processing task for details.

Visualize post-processed data

An example Jupyter notebook panhtc_points_post_process.ipynb has already been included. Run it through terminal:

>jupyter notebook panhtc_points_post_process.ipynb

Run through the notebook, and click “Display/Refresh” button, an interactive plot will be displayed:

Jupyter notebook for point HTC

Select tabs on the plot to display the correlation between properties and composition

Jupyter notebook for point HTC: composition vs properties Jupyter notebook for point HTC: properties

Select ranges of properties and display corresponding composition range. For example, set volume fraction of Fcc > 50%, and click “Display/Refresh” button, the plot will be updated to:

Jupyter notebook for point HTC: updated Fcc range

Change combination of properties, select ranges of properties, and display corresponding composition range. For example, set volume fraction of Fcc > 50% and Diamond < 50%, and click “Display/Refresh” button, the plot will be updated to:

Jupyter notebook for point HTC: updated Fcc and Diamond ranges

The inputs and outputs of this post-processing task

The following figure displays inputs and outputs of this post-processing task.

Post-Processing diagram
  • File structure

    The follow diagram displays the file structure used by this post-processing task:

    Note

    The “resource” folder is a copy of Pandat workspace. See details of Pandat workspace in Chapter 2.1 of Pandat Manual: .

    panhtc_points_post_process
     |- output  # outputs
     |    └─ merged.csv  # A CSV file containing results after post-processing
     └─ resource  # a copy of Pandat workspace folder
          └─ default # The parent folder containing all HTC points
              └─ Point Calculation_X # The X-th child folder of point calculation
                     └─ Table    # The folder containing .table file
                         └─ Default.table    # the .table file from a Pandat calculation
    
  • Inputs

    The input files are under /resource folder.

    • resource/default/Point Calculation_X/Table/Default.table: A Pandat .table file containing a point calculation result

  • Outputs

    The output files are under /output folder.

    • merged.csv: A CSV file containing results after post-processing.

  • merged.csv

    Piont HTC results are combined into the merged.csv file, with an additional “task_id” column as index.