I’m experiencing a problem with submitting my results to the platform. I do the following:
I downloaded and unzipped test data
I use your starter notebook and run the cell which creates predictions with path to the test data
Then, I run “create_submission” with these predictions
I download this files, try to submit and … get “Error: seismic-92515931.npy-i_0 and target-92515931.npy-i_0 have different shapes”
I have compared it to your test submission and realized that shapes were indeed wrong. However, there are several things that unclear to me:
Test does not specify which patch we should predict on which means that targets created on our own are always different from ones specified by you and the score will always be close to zero.
Do we need to have the same file names in our submission as in the sample submission. The sample submission uses “target” instead of “seismic”. The secind one is from names of files in the test.
I’m pretty sure that I misunderstood something and will be happy to get some help with it. Thanks!
Thanks for the question @sanektim! Hopefully we can explain a bit further.
The test data that has been provided will have a 25% block of the volume set to nan. In the training data we provide both the volume with a missing block of data and the missing block of data. Your job is to predict the missing block of data in the test dataset. Your output for the submission file generator should have the same shape as the missing blocks of data in the test set. When you run this through the submission file generator, it will select 3 inlines and 3 crosslines for SSIM scoring. You will need to predict the full 3D missing section, as you do not know where the submission code will extract.
Please keep name the same as the example in the starter notebook (target_<volume_id here>.npy).
HI Guys
I am exasperated. I do not see how the shape of my submission can possibly be different from the original. I took the target array and filled with a dummy value as an experiment on the format.
My target shape: (75, 300, 1259) Again, this is the original from training_data_generator
My first parameter, the name of the source volume: “example_seis_vol_2.npy”
cheers, Eric
I have this error “Error: seismic-92515353.npy-i_0 and target-92515931.npy-i_0 have different shapes”
However I check and check and I’m sure that it’s the same.
The shape of the targets will be either (300,75,1259) or (75,300,1259)
Your prediction volumes should match the shape of the missing blocks of data in the test data
The test data are (300,300,1259) seismic volumes that contain a block of data of size (300,75,1259) or (75,300,1259) set to nan
Check to make sure you are creating the submission file correctly. create_submission() will require three inputs in order to create a correct submission format; seismic_filenames is a list of the test data filenames, predictions is a list of the target Numpy arrays (what your model is predicting), and submission_path is the local location the submission file will be saved.
The create_submission() function works by extracting 6 2D slices from each of the target volumes, so the submission file shapes are not equal to the target file shapes.
To help with this, please use this code snippet below to help identify target areas and shapes in the test volumes:
def get_mask(seismic):
“”"
Function for mask extraction from volume with deleted patch.
Parameters:
seismic: np.ndarray 3D matrix of survey with deleted patch.
Returns:
target_mask: np.ndarray, position of target 3D matrix inside seismic 3D matrix.
"""
target_mask = np.zeros(seismic.shape).astype('bool')
nans = np.argwhere(np.isnan(seismic[:, :, :]))
indices = [(nans[:, i].min(), nans[:, i].max()+1) for i in range(len(seismic.shape))]
target_mask[indices[0][0]:indices[0][1], indices[1][0]:indices[1][1], indices[2][0]:indices[2][1]] = True
return target_mask
Hello!
I encounter the following error and cannot understand why something is wrong. This error says that different test targets have different shapes, but the initial test volumes have different orientations along which the patches with nan are. Therefore, the shapes of the targets are (300, 75, 1259) and (75, 300, 1259), respectively.
So, does this error imply that I need to transpose xlines targets and make all the targets be of shape (75, 300, 1259)? Or I misunderstood something?
Thank you so much in advance!
I had the same issue. It looks like the order of slices in the submission file is important. I ordered it in the similar to the sample submission file and it was accepted.
Thank you Dmitry!! I have been having a terrible time here. Why the order of the file names should matter seems strange. I had assumed that the names were a key to the values of the correct slices. It is only a little progress. I am no longer getting the bad shape message. It seems I crashed the scoring algorithm.
Anyway, for anyone still struggling here, this file order seems to work:
file_names = [‘seismic-92515355.npy’,
‘seismic-92515931.npy’,
‘seismic-92551495.npy’,
‘seismic-92515357.npy’,
‘seismic-92508315.npy’,
‘seismic-92515353.npy’,
‘seismic-92515932.npy’,
‘seismic-92568236.npy’,
‘seismic-92551498.npy’,
‘seismic-92568004.npy’,
‘seismic-92551496.npy’,
‘seismic-92568235.npy’,
‘seismic-92515930.npy’,
‘seismic-92515356.npy’,
‘seismic-92551497.npy’]
I don’t think getting this sequence right is appropriate for a contest of this difficulty, The number of possible orderings is 15!, 15 factorial.
cheers, Eric
Hi Guys
I tried my submission again today with the same error message:
Scoring algorithm failed
This reminds me of my intro to programming, a 1/2 unit class in Fortran. This was in the era of punch cards. I would labor over my program and get back the cryptic message: syntax error
This 1/2 unit course cost me as much time as any of my other courses.
This was only a dummy submission to test the format, and had nothing to do with the actual goal of the contest. The patch was filled with a constant. I did have a thought. At any point does the scoring invert a matrix? A matrix with identical rows will be singular.
Shorts weather today here in Bishop.
cheers, Eric
It appears that the right order for the keys comes from the ordering in the sample-submission provided. The file listing order can be populated by:
“”"
get the right order from the sample-submission provided
sample_sub = np.load(‘…/patch-the-planet-sample-submission.npz’, allow_pickle=True)
target_order =
for filename in list(sample_sub.keys()):
filename = filename.split(‘.’)[0].replace(‘target’, ‘seismic’) + ‘.npy’
if filename not in target_order:
target_order.append(filename)
“”"
however, I also got the new error message “scoring algorithm failed”
This will not be an issue for Final Submissions on the hold out data. For the Predictive Leaderboard there are a couple of options to verify the slice submission order listed above.
Please check the .npz format for your submission and ensure that it matches the sample submission example provided with the starter notebook for this challenge.
