Fig. 1

(A Neural network schematic) From left to right: a pair of individuals is selected for the feature-extraction step—this will be repeated for \(k_{\text{extract}}\) pairs. The genotype matrix shows the genotypes for the pair. Cream colored tensors are the output from convolution layers. The blue box over the genotypes shows the convolution kernel for the first layer. Red tensors are the output from pooling layers. The spatial coordinates for the current pair are subsetted from the locations table (Input 2). The Euclidean distance is concatenated with the flattened convolution output. Green tensors are the output from flattening, concatenating, or dense layers. The extractor is repeated for \(k_\text{extract}\) different pairs of individuals, and the resulting features are concatenated together. The dimensions noted beneath each tensor will vary depending on the input size; this example uses 5000 SNPs (although the image of the genotypes shows a smaller number of SNPs). The visualized size of each tensor is proportional to the square root of the actual dimensions. Neural network images were generated using PlotNeuralNet (https://github.com/HarisIqbal88/PlotNeuralNet). (B Box plots) Also shown are validation results using Rousset’s method (dark grey), disperseNN (light grey), and disperseNN2 (white), with two different sample sizes. Outliers are excluded