Parallel High-Throughput Point Calculation (HTC)

This example demonstrates how to use PanPython SDK to perform Parallel HTC for several batches of points concurrently.

Post-processing and visualization of the HTC results will be discussed in Post-Processing of data from Point HTC.

About point HTC

Point HTC allows a user to perform calculations for numerous compositions and/or temperatures in the user-defined state space by a simple setting. Alloy compositions and/or temperatures that satisfy user-defined criteria can then be discovered through data mining of the simulated results.

Step-by-step guide

Note

A video guide will be available very soon.

  1. Open the demo folder

    “/Solution Examples/parallel_htc_points/” from PyCharm IDE. See Installation to setup the IDE environment.

  2. Build a task script

    An example script parallel_htc_points.py has already been include. Following the steps the create your script:

    Import packages
    import os
import socket
import multiprocessing as mp
from panpython.system import System
from panpython.task.htc import HtcMesh
import pickle
    Setup Pandat.exe path:
    pandat = "YOUR_PATH_TO_PANDAT_EXE/Pandat.exe"
    Setup working folders and input files:
    dir_name = os.path.abspath(__file__)
file_name = os.path.basename(__file__)
task_path = dir_name[:-len(file_name)]

dump_path = os.path.join(task_path, "output")
batch_file = os.path.join(task_path, "resource", "Point_AlMgSi_400C.pbfx")
config_file = os.path.join(task_path, "resource", "Input_point.JSON")

    Note

    The calculation type is set in Input_point.json:

    {
   "Type": "point",
}

    The calculation type in the batch file template must be consistent with this type, or an exception will be raised:

    <calculation name="Point Calculation_Point_Al-Mg-Si_400C" type="point">

    Note

    The tdb file is set in the batch file template Point_AlMgSi_400C.pbfx:

    <databases>
 <database type="tdb" file_name="AlMgSi.tdb"/>
</databases>
    Set thread(s) number:
    #################
# set thread number
# note: if your number is bigger than system capability, the number will set to max
# available thread number on your system
thread_number: int = 4
print('>> Max thread  number on machine: ', socket.gethostname(), ' is ', mp.cpu_count())
if thread_number > mp.cpu_count():
    thread_number = mp.cpu_count()
print('>> Thread number is set to ', thread_number)
    Initialize the HTC task instance:
    #################
m_system = System(pandat=pandat, dump_path=dump_path)

##################
m_system.add_task(task_definition=HtcMesh(batch_file=batch_file, config_file=config_file, thread_num=thread_number))
    Run HTC task instance in multi-thread mode:
    ##################
if __name__ == '__main__':
    mp.freeze_support()  # required to run multi-thread
    m_system.run()
  3. Run the script

    After the HTC is finished, tables and generated tables/files are ready to use. See Inputs and outputs of this HTC task for details.

Inputs and outputs of this HTC task

The following figure displays inputs and outputs of this HTC task.

cHTC diagram
  • File structure

    The follow diagram displays the file structure used by this HTC task:

parallel_htc_points
 |- output  # outputs
 |    |- intermediate
 |    |   |- *.dat    # Result Tables in csv format
 |    |   └─ *.table  # Result Tables
 |    |
 |    |- failed_calculation.json  # Record of failed calculations
 |    |- PanHTC_interface.txt     # Pandat HTC Alloy Table
 |    └─ task_id.txt              # Condition Dictionary
 └─ resource  # inputs
      |- AlMgSi.tdb              # Database to be used by HTC calculation
      |- Input_point.json        # A Config. file (JSON) defining an HTC mesh grid
      └─ Point_AlMgSi_400C.pbfx  # A template of batch file used calculation
  • Inputs

    The input files are under /resource folder.

    • Configuration file of point HTC: A JSON file defining an HTC mesh grid. The file name is /resource/Input_point.json.

    • Database: A .tdb or .pdb file to be used by HTC calculation. The file name is /resource/*.tdb or /resource/*.pdb.

    • Batch file: A template of batch file used for Pandat calculations in HTC. The file name is /resource/*.pbfx.

  • Outputs

    The output files are under /output folder.

    • Result Tables: The original tables from Pandat. The tables are stored under /output/intermediate/ folder.

    • Record of HTC conditions: A JSON-styled dictionary containing all calculation conditions of HTC. The file name is output/task_id.txt.

    • Pandat HTC Alloy Table: This file can be imported by Pandat GUI to perform an HTC calculation with the identical condition. The file name is output/PanHTC_interface.txt.

    • Failed Calculations: a record of failed calculations. The file name is output/failed_calculation.txt.

Configuration file of point HTC

One of the input files. The file name is /resource/Input_point.json.

A JSON file defining an HTC mesh grid. An example of Config. file as an input to HTC:

{
   "Type": "point",

   "Units": {
     "n": "x%",
     "T": "C"
   },
   "T": {
     "start": 400,
     "end": 400,
     "step size": 0
   },
   "Al": {
     "start": 0,
     "end": 100,
     "step size": -1
   },
   "Mg": {
     "start": 10,
     "end": 50,
     "step size": 10
   },
   "Si": {
     "start": 10,
     "end": 50,
     "step size": 10
   }
}

The keys of the JSON file is defined in the following table:

Key

Definition

Type

Calculation type of Pandat

Units

Units following Pandat’s convention

T

Temperature grid

Al

Aluminum composition grid

Mg

Magnesium composition grid

Si

Silicon composition grid

The properties of each key’s value is defined in the following table:

Property

Definition

start

start value of a grid

end

end value of a grid

step size

step size of grid; If its key is an element and set -1, its key is the balance element

Record of HTC conditions

One of the output files. The file name is output/task_id.txt.

A JSON-styled dictionary containing all calculation conditions of HTC. An example of Condition Dictionary as an output from HTC:

{
     "0": {
         "T": 400,
         "Al": 80.0,
         "Mg": 10,
         "Si": 10
     },
     "1": {
         "T": 400,
         "Al": 70.0,
         "Mg": 10,
         "Si": 20
     },
     "2": {
         "T": 400,
         "Al": 60.0,
         "Mg": 10,
         "Si": 30
     }
 }

Keys of the JSON file is the id of a calculation. Properties of the calculation include all the keys from Config. file.

  • Pandat HTC Alloy Table

    This is one of the output files. The file name is output/PanHTC_interface.txt. This file can be imported by Pandat GUI to perform an HTC calculation with the identical condition.

    See Generate an Table of Alloy Compositions for HTC in Pandat GUI for the details and instructions on the usage of this interface.