The 3DPlan algorithm was developed during my diploma thesis, entitled "Automated detection of edges in point clouds using semantic information", at the School of Rural and Surveying Engineering of the National Technical University of Athens, which is available at ThodorisBetsas_DiplomaThesis. At first, the 3DPlan algorithm enriches the given images with a new channel including the edge semantic information. Thus the three-channel images (RGB) are transformed into four-channel (RGBL). Afterwards, the four-channel images are inserted into an SfM-MVS software to produce initially a sparse and finally, a dense point cloud which is enriched with the edge semantic information.

For this purpose, three approaches were developed.

    1) Using the OpenSfM software which was firstly modified to manipulate four-channel images.
    2) Using the Agisoft-Metashape software with its GUI.
    3) Using the Agisoft_Metashape software with the provided python module.

The points belonging to edges are firstly detected, into the semantically enriched point cloud and then classified into points of each edge. Finally, each segment of points is vectorized and thus, the approximated 3D plan of the object of interest is produced as "3DPlan.dxf".

numpy == 1.19.2
opencv-python == 3.4.8.29
dxf == 1.1.1
pathlib == 1.0.1
imutils == 0.5.3
metashape == 0.0.4 (Optional for the approach numb. 3)
Pillow == 7.0.0
sklearn == 0.0
scikit-image == 0.17.2

1) Clone/Download the 3DPlan master.
2) Add the RGB images into "rgb" directory.
3) Edge semantic information.
    3.1) Available:
        Add the semantic channel i.e., 1D image, for each image, into "semantic_images" directory.
        The semantic channels must be named as "rgbimagename_l.jpg"
        For example: RGB image --> 4G0R6560.JPG
                     semantic channel --> 4G0R6560_l.jpg
    3.2) Not available:
        The 3DPlan algorithm contains a live-implementation of the Canny algorithm which could be used for producing
        the edge semantic information.
        The live-editor begins and the user modifies Canny's parameters. When the user is satisfied by the detected
        edges, he presses the Q button to terminate the editing procedure.
        Then the 4D image is produced automatically and saved into "images" directory which is created automatically.
        This procedure is executed for each RGB image.
        (Nothing to do right now, go to step 4)
4) Execute the "3DPlan.py".

(Question 1) dd:mm:yy hr:min:sec, 3D (0) or 4D (1) output? (Write 0 or 1): The recommended answer is "1". (Question 2) dd:mm:yy hr:min:sec, Agisoft-Metashape (0), Mapillary-OpenSFM (1) or MyTriangulation (2)? (Write 0, 1 or 2):

(Question 2.1) dd:mm:yy hr:min:sec, Python Module (0) or GUI (1) output? (Write 0 or 1):
Select how you will apply the Agisoft-Metashape pipeline. 
The python-module choice executes the pipeline automatically.
On the other hand the GUI option waits for the user to run the algorithm using the Agisoft-Metashape
graphical user interface. When the dense point cloud is produced the user must save it into the 
"Lines" directory, which is produced automatically, as "merged.txt".

The 3DPlan algorithm is executed automatically.

If the edge semantic information is not available the user should select the "Canny" choice i.e., write 0 and
press "enter". Then the 3.2 step (How to use) is executed.
If the edge semantic information is available the user should select the "semantic information" choice i.e., write 1
and press "enter". (! Notice !) Look the step 3.1 (How to use).

The user should write the images format, which are stored into "rgb" directory, for example .JPG
and the to press "enter". The valid answers are '.JPG', '.jpg', '.TIFF', '.tiff', '.tif',
'.PNG', '.png'.

The 3DPlan algorithm is executed according to the answers of the user and the 3DPlan.dxf archive is produced into the "Lines" directory.

1. LINE DETECTION USING MANUALLY ANNOTATED IMAGES AND THE AGISOFT-METASHAPE SOFTWARE.
   The two images which are inserted into the "rgb" directory:
   
   

The two manually annotated images which are used for this implementation and inserted into "semantic_images" directory:

Then two masks are produced automatically by the 3DPLan algorithm, using the manually annotated images. Afterwards the 4D images are created automatically by the 3DPLan algorithm and fed into the SfM-MVS algorithm.

The produced dense point cloud is visualized using the RGB colors and the labels ones.

Additionally the detected and vectorized edges are displayed in the images below.

A close view of the vectorized line in combination with the rgb dense point cloud is depicted in the image below.

2. MULTIPLE LINES DETECTION USING THE CUNNY ALGORITHM AND THE AGISOFT-METASHAPE SOFTWARE.
   The processing is exactly the same as the previous example. The difference is that a live-canny editor is executed automatically
   and the user creates the edge semantic information. When the user is satisfied with the edge map press the "Q" button and continues
   to the next image. ( Notice !) Look the step 3.2 (How to use).
   

The live-canny editor:

The detected edges:

The extracted 3D vectors, i.e. "lines.dxf", in combination with the sparse point cloud:

An in depth analysis of the proposed approach and in general of the 3DPlan algorithm is available at ThodorisBetsas_DiplomaThesis.

3DPlan is GPL-3.0 licensed, as found in the LICENSE file.