3D ML. Part 5: convolutions on graphs

, , “” — . , , , .

, .

3D ML :

1. 3D
2. 3D ML
3. 3D ML

GitHub .

IT- “VR/AR & AI” — PHYGITALISM.

?

, . , , , , , , .

, , , .

, “Representation Learning on Graphs: Methods and Applications” [1]. , Stanford CS224W: Machine Learning with Graphs. Sberloga, , , . ODS, (.. ).

Medium:

1. How to do Deep Learning on Graphs with Graph Convolutional Networks Tobias Skovgaard Jepsen;
2. - A Gentle Introduction to Graph Neural Networks (Basics, DeepWalk, and GraphSage) - Hands-on Graph Neural Networks with PyTorch & PyTorch Geometric Kung-Hsiang, Huang (Steeve);
3. Jan H. Jensen ( );
4. Medium — .

3D ML , , . — , (. [15]), , , (. [14]). , 3D ML, .

, , , ( python rdkit python NetworkX). , PyTorch Geometric ( , , ). TensorFlow Graphics “Semantic mesh segmentation”, , , . 3D ML .

, 4, , .

.1 [2]: , (Euclidean CNN) , , , (Geometric CNN, ).

— , “” () . , , ( ). , , , . , , .

, “Geometric deep learning: going beyond Euclidean data” [2] ( 3D ML, , ).

:

• (shift-invariance): (, ) , , . : .
• (locality): ( , ). , , .
• (compositionality or hierarchy): , , “ ” ( , : , , ..).

. — ( ).

, , ( ) ( ).

.2 : MNIST; : . (. [3]).

, , , , . python NetworkX. .

:

import networkx as nx

G = nx.grid_graph([4, 4])
nx.draw(G)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, .

, $$$W_ {3 \ times 3}$ . , , — $$$X_ {ij}$ . (dot product) ( ), (. ). “A guide to convolution arithmetic for deep learning” [4].

, , . , , , , .

, , , — ( , , 9 ). .. pooling . , , ,

$$

$$X_1 W_1 + X_2 W_2 \ ne X_2 W_1 + X_1 W_2$$

, . , - .. “” . .

.3 [5].

. : — ( — $$$V$ , — $$$G$ ), — . , ( ) , — .

- , , . — (averaging) [6] (summation) [7] , $$$W$ . , . [8].

5 (.4). $$$X_1$ :

$$

$$X_1 = (X_1 + X_2 + X_3 + X_4) W_1.$$

, , , , .

.4 $$$X_1$ .

“” . , . , .

, ( MNIST). $$$X ^ {(l)}$ — $$$l$ - , $$$W ^ {(l)}$ — $$$l$ - . :

$$

$$X ^ {(l + 1)} = X ^ {(l)} W ^ {(l)}.$$

, ( ), , MNIST , 91% ( ).

, ( — ). — $$$A$ . . , .

.5 5 ( — 0, — 1) .

, $$$28 \ times 28$ MNIST. numpy:

import numpy as np
from scipy.spatial.distance import cdist
img_size = 28  # MNIST image width and height
col, row = np.meshgrid(np.arange(img_size), np.arange(img_size))
coord = np.stack((col, row), axis=2).reshape(-1, 2) / img_size
dist = cdist(coord, coord)  # . 6 ( )
sigma = 0.2 * np.pi  # width of a Gaussian
A = np.exp(-dist ** 2 / sigma ** 2) # .6 (  )
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, dist



— $$$28 ^ 2$ , $$$A$ — ( , , ). , , — , (. [2, 9]).

.6 -: , , $$$28 \ times 28$ .

, $$$A$ , .. $$$\ mathcal {A}$ . , :

1. $$$\ mathcal {A} = A$ $$$\ mathcal {A} X ^ {(l)}$ [7];
2. $$$\ mathcal {A} = \ frac {A} {\ | A \ |}$ $$$\ mathcal {A} X ^ {(l)}$ [6].

, , :

$$

$$X ^ {(l + 1)} = \ mathcal {A} X ^ {(l)} W ^ {(l)}.$$

, .

PyTorch, :

import torch
import torch.nn as nn

C = 2  # Input feature dimensionality
F = 8  # Output feature dimensionality
W = nn.Linear(in_features=C, out_features=F)  # Trainable weights

# Fully connected layer
X = torch.randn(1, C)  # Input features
Z = W(X)  # Output features : torch.Size([1, 8])

#Graph Neural Network layer
N = 6  # Number of nodes in a graph
X = torch.randn(N, C)  # Input feature
A = torch.rand(N, N)  # Adjacency matrix (edges of a graph)
Z = W(torch.mm(A, X))  # Output features: torch.Size([6, 8])
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

. , MNIST , , . , , , , $$$\ mathcal {A} X ^ {(l)}$ , , , . , , [10, 11, 12].

.7 , , () ().

, , :

import torch.nn as nn           # using PyTorch
nn.Sequential(nn.Linear(4, 64), # map coordinates to a hidden layer
             nn.ReLU(),         # nonlinearity
             nn.Linear(64, 1),  # map hidden representation to edge
             nn.Tanh())         # squash edge values to [-1, 1]
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

.8 2D- , , . (, 92,24%), (, 91,05%), (, 92,39%).

, , ( ), . , , : , , , . (, , ), , [13].

(GNN) — , . , . GNN — . , GNN .

.9 GNN “Anisotropic, Dynamic, Spectral and Multiscale Filters Defined on Graphs”.

, .

, , [6].

, Medium.