How to save values to an array?

How do I access the values of these MATLAB variables based on a matrix which stores the indices of the required columns?

• If there's anything you think is way too complicated please comment so that I can explain clearly. Any help would be appreciated a lot. It would speed up my work significantly. This is going to be slightly complicated. Let's start with describing the data. I have 4 arrays with 18 structs. Let's call them S1, S2, S3, S4. And from each struct I need to extract and store the values of a particular member. From each of these members I only need to extract specific columns. The indices of these columns are given in the matrix I mention. So let's access these members as S1{1}.val(). The format of the matrix (called is M) as follows - If you check this image. Column 1 and Column 2 have the start and stop values. Column 3 and 4 also have start and stop values but for some other data but corresponding to the same array (out of S1 , S2, S3, S4) and this repeats for all the columns. These start and stop values are actually indices that I require to be able to extract corresponding values from the structs. Example - S1{1}.val(:,M(1,1):M(1,2)) But you notice the NaNs, right? So that means that for S1, I need to save values corresponding to the rows 1 to 4, for the first 6 columns. and for remaining 6 columns I need to store values corresponding to the rows 1 to 3. And for each [row,[column(i):column(i+1)] ] I need to store the data from S1 in a variable (or a struct of these variables or a cell array, which ever you might think convenient). And every column pair like [C1,C2] [C3.C4] etc.. corresponds to different data but saved in the same struct members. So saving them in similar variables would be much helpful. For example, a cell array of structs corresponding to the same data - V.data1 = {S1.val(:,M(1,1):M(1,2)),S1.val(:,M(2,1):M(2,2))..S1.val(:,M(4,1):M(4,2)),S2.val(:,M(6,1):M(6,2)),..S2.val(:,M(12,1):M(12,2)),S3 similar to S1, S4 similar to S2} %For subsequent variables data changes corresponding to columns V.data2 = {S1.val(:,M(1,3):M(1,4)),S1.val(:,M(2,3):M(2,4))..S1.val(:,M(4,3):M(4,4)),S2.val(:,M(6,3):M(6,4)),..S2.val(:,M(12,3):M(12,4)),S3 similar to S1, S4 similar to S2} As you see ,the above repeats for S3 which are given by rows 14 to 17. Similar for S2 and S4. But for S2 and S4 I have values corresponding to 7 rows  (for all columns in this case). These rows are given by 6 to 12 and 19 to 25.  Oh and the above says S1, S2 and such instead S1{1}, S1{2}... to indicate that I wish to do the same for all the structs and their corresponding members. Now how do I save all of this? It's bugging me and I would like a clear and efficient way. Any suggestions?

• Answer:

  Your problem description is that you have a complex hierarchical data type (matlab struct class array) and you need to extract data from it according to some rules while using keys encoded in your matrix M. There are two parts to your problem. The first one is the automated handling of the M matrix (extracting index values from M) while following class and type rules along with rules encoded in M. The second one is extracting data from your source S1, S2, S3 and S4 matrices using an approach convenient for analysis/efficient manipulation. I describe two approaches to implement the first part of automated M handling here. I first use a standard indicial book keeping approach. Then I show a more elegant alternative based on converting the matrix M to an image, then analyzing that. The image based analysis approach is more efficient and intuitive. (Your M matrix was transcribed from the image on this page using OCR so it might not match up with your actual matrix.) Ms= [4x2] [4x2] [4x2] [3x2] [3x2] [3x2] | (data from S1) --+-----+----+---------+-----+-----+---| [7x2] [7x2] [7x2] [7x2] [7x2] [7x2] | (data from S2) --+-----+----+---------+-----+-----+---| [4x2] [4x2] [4x2] [3x2] [3x2] [3x2] | (data from S3) --+-----+----+---------+-----+-----+---| [7x2] [7x2] [7x2] [7x2] [7x2] [7x2] | (data from S4) --=-----=-----=--------=-----=-----=---| dat1 dat2 dat3 dat4 dat5 dat6 | ---------------------------------------| Based on your description, you have four data sources and six interpretations of the data obtained by mixing data from the sources according to indicial rules. The indicial rules are given in the M matrix. My first step is to cast M into submatrices that simplify the description. I interpret my submatrices as shown above. So I need to reduce M into 24 submatrices. A description of a submatrix in a larger matrix requires four numbers/values/points - a reference location, a width and a height value. For example we can assume that these four points describe the top-left corner (r0, c0) and the bottom-right corner (r1, c1) of each submatrix. These points are defined with respect to the top-left corner of M (the (1,1) position). This allows the 4x6 submatrix array to be described by 4 sets of 4x6 arrays called r0, c0, r1 and c1, and gives us a systematic approach to collecting index values from matrix M. Standard book-keeping approachc0 and c1 are easy to calculate (AP). To calculate r0 I create two matrices, one called nlines, the other the shift matrix. The nlines matrix stores number of rows (4,7,4,7) for Data1/2 and (3,7,3,7) for Data3/4. The shift matrix stores the offsets introduced by NaN (You can see here that I treat my matrices like images and elements like pixels). We know the initial values of r0 (r0(1,:)=ones(1,6)) and final values of r1 (r1(4,:)=25*ones(1)). So we need to go down to calculate r0, and go up from bottom to calculate r1. This approach is best understood reading the code. Interpreting M as an image (requires matlab image processing toolbox) M is a 25x12 array and our goal is still to figure out r0, c0, r1, c1 efficiently. The NaN values in M represent markers segregating four regions in the array. The grayscale image below shows a normalized M. We can convert it into a logical matrix where each element is the logical result of ~isnan. This binary matrix is shown below. The binary image shows that the 'components' that are not 'connected' (the white blobs that don't touch or have adjacent pixels), each represent 6 submatrices of M. So I do a connected components analysis and figure out which pixel groups are associated with which of the four blobs. The individual pixel groups are shown highlighted by a different color in the image titled 'connected component from BW'. Since we already know that a submatrix can only have 2 columns, we can divide/segment these blobs into groups of 6. This is shown below, image to right. The image has 24 colored boxes that represent our submatrices. We also know all the pixels that belong to each of these submatrices. So r0, r1, c0 and c1 are the least row, highest row, least column and highest column index in each of the 24 groups. That is the second approach to determining the submatrices. So once we determine the set of four points, we need to fetch the index values stored in M and store them somewhere. I create a cell array of size 4x6 called BigS (for Big Store) and store all the indices from each of the 24 submatrices in there. BigS(i,j) let's you easily categorize or call upon any dataset that you need. Now I don't know what analyses you are interested in. Every method to fetch data from Si can result in loss of information. But I am going to assume you know what you are doing, and use a naive approach where I discard all information about the original 18 structures during retrieval. In the following code, I create a prototype of the struct arrays, S1.. S4. The contents of the arrays are displayed in the command window. Version 2.a - Book keeping approach function [Data1,Data2,Data3,Data4,Data5,Data6]=junk %% Bookkeeping close all; clear all; clc % Read the image file; LOCAL FUNCTION. Not available on your Matlab install % p=xsnap(1); % Take desktop snapshot % M=xread(p); % Read text using Tesseract % Define M. There might be OCR related noise in the transcribed nums M=uint16([960 1340 3040 3500 4990 5580 6950 7555 8850 9355 11025 11480;13010 13440 15000 15535 17055 17640 18970 19625 21020 21590 23035 23520;25050 25425 27035 27550 28990 29575 31055 31630 33010 33570 35010 35405; 37060 37465 39020 39545 41060 41720 NaN NaN NaN NaN NaN NaN;NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN;1010 1470 2060 2535 3045 3620 4045 4600 5035 5550 5990 6460; 7085 7570 8010 8500 9085 9640 10055 10595 11020 11580 12010 12510;13080 13535 14030 14590 15065 15625 16040 16595 17045 17575 18025 18490;19065 19450 20065 20520 21000 21540 22005 22570 23030 23555 23995 24465; 25080 25565 26060 26565 27105 27680 28025 28565 29030 29540 29975 30415;30995 31365 31985 32480 33015 33550 33980 34555 34980 35465 35940 36465;37020 37430 38020 38475 38990 39495 40005 40470 40955 41455 41920 42305; NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN;955 1475 3010 3565 5045 5555 7030 7560 8980 9610 10945 11525;13100 13780 14990 15435 16965 17535 18990 19520 20995 21545 22985 23535; 25035 25570 27010 27530 28985 29455 30955 31435 32995 33550 35005 35605;37050 37685 38985 39550 40995 41540 NaN NaN NaN NaN NaN NaN;NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN; 935 1375 1980 2495 3005 3580 3960 4595 5045 5545 5980 6405;6940 7335 7960 8405 8970 9485 9970 10570 11025 11520 11950 12400;12985 13430 13895 14385 14835 15405 15855 16445 16870 17415 17930 18350; 19000 19420 19965 10455 21045 21600 21950 22500 22950 23500 23915 24320;24900 25355 25850 16335 26895 27455 27885 28450 28965 29490 29900 30385;30960 31360 31845 32345 32925 33515 33940 34550 34975 35465 35890 36390;36965 37360 37900 38380 38935 39515 39955 40530 40950 41525 41945 42380]); %% Generate a prototype for S1, S2, S3, S4 % Si is an array of 18 structs. % Each struct in Si has 13 fields in all. 13=My random choice. % Each struct in Si has 'val' as the 13th field. % Si.val has 18 3xN matrices (It points to all '.val's in Si) % All fields except 'val' have random logical matrices populating them % (can be empty) % I use N=42380 since max(M(:))=42380; I use logical values for data since % it uses 8bits, and double uses 8bytes (64b), that times 18 time N is a % mouthful. If N is a variable, then matrix M controls which indices of % S1{i}.val(:,x:y) you ask for (since x:y originate in M). So you will be % fine with whatever Ni you have. [S1, S2, S3, S4]=makeUpRandomCrap(42380,true); %% IMPLEMENTATION % NLINES MATRIX, describes number of rows each submat of Ms describes % Can also be viewed as the main proxy for Ms. You will see me use this % throughout to indicate counting bounds. n=[4 4 4 3 3 3; 7*ones(1,6); 4 4 4 3 3 3; 7*ones(1,6)]; numCol=2; % Number of columns in each submatrix % SHIFT MATRIX, to account for variable thickness/linewidth of NaN lines % There are effectively three lines segratating M horizontally. These pesky % lines vary in thickness causing grief to counting strategies. s=[1 1 1 2 2 2; 1 1 1 1 1 1; 1 1 1 2 2 2; 1 1 1 1 1 1]; % Top left corner coordinates of each submatrix = (r0,c0) [ordered pair] c0=ones(size(n,1),1)*[1:numCol:size(M,2)-1]; % skip cols by numcols r0(1,:)=ones(size(n(1,:))); % Origins Matrix, define the first row for i=2:size(n,1), % begin at second row since the first one is already defined r0(i,:)=r0(i-1,:)+n(i-1,:)+s(i-1,:); % X coordinate of top left corners of each submatrix end; % Bottom right corner coordinates of each submatrix = (r1,c1) [ordered pair] c1=ones(size(n,1),1)*[(1+numCol-1):numCol:size(M,2)]; % Skip cols by numcols-1 r1=zeros(size(n)); % Since we cycle back from last row to first, we need y1 mat defind r1(end,:)=size(M,1)*ones(size(n(1,:))); % Define the last row for i=(size(n,1)-1):-1:1 % Cycle back from last but one row, since lar row is alredy defined r1(i,:)=r1(i+1,:)-n(i+1,:)-s(i,:); end for i=1:size(n,1), % Cycle through Si for j=1:size(n,2) % Cycle through dataj indexS=[]; % Create temporary index store for k=1:n(i,j) % Step through the rows in M indexS=[ indexS [M( r0(i,j)+k-1 , c0(i,j) ):(1):M( r0(i,j)+k-1 , c1(i,j) )] ]; % Collect indices, append set (Set-Union) end BigS(i,j)={indexS}; % Store final appended set in BigS. Dataj is made of Si with i=1:4, Si contributes Dataj with j=1:6 end; end; %% FETCH DATA % Now I don't know what analyses you are interested in. Every method to % fetch data from Si can result in loss of information. But I am going to % assume you know what you are doing, and use a naive approach where I % discard all information about the original 18 structures during % retrieval. % The template for extracting data is Si(k).val(:,BigS{i,j}) for i=1:4, for j=1:6, for k=1:18, % Store in individual numeric arrays, an 3xNx18 cuboid called % DatajSi,where j=1:6 and i=1:4. These represent blocks of jth % data contributed by ith Si. eval(['Data' num2str(j) 'S' num2str(i) '(:,:,k)= S' num2str(i) '(' num2str(k) ').val(:,BigS{' num2str(i) ',' num2str(j) '});']); end end end %% DatajSi are the following definitions are ALL simple mxArrays % that take less space than cell/struct. cell/structs also lead to memfrag % and in general mem(struct)>>mem(cell)>simple arrays. Unless there's mixed % types, there's no good reason to use structs. % Here's one easy way to store stuff. Data1=[Data1S1 Data1S2 Data1S3 Data1S4]; % size(Data1)=[3 9982 18] Data2=[Data2S1 Data2S2 Data2S3 Data2S4]; % size(Data2)=[3 10005 18] Data3=[Data3S1 Data3S2 Data3S3 Data3S4]; % size(Data3)=[3 12327 18] Data4=[Data4S1 Data4S2 Data4S3 Data4S4]; % size(Data4)=[3 11315 18] Data5=[Data5S1 Data5S2 Data5S3 Data5S4]; % size(Data5)=[3 10695 18] Data6=[Data6S1 Data6S2 Data6S3 Data6S4]; % size(Data6)=[3 9460 18] % If you want all Dataj to be a 3XN matrix, losing all struct info Data1=reshape(Data1,size(Data1,1),size(Data1,2)*size(Data1,3)); % size(Data1)=[3 179676]; Data2=reshape(Data2,size(Data2,1),size(Data2,2)*size(Data2,3)); % size(Data2)=[3 180090]; Data3=reshape(Data3,size(Data3,1),size(Data3,2)*size(Data3,3)); % size(Data3)=[3 221886]; Data4=reshape(Data4,size(Data4,1),size(Data4,2)*size(Data4,3)); % size(Data4)=[3 203670]; Data5=reshape(Data5,size(Data5,1),size(Data5,2)*size(Data5,3)); % size(Data5)=[3 192510]; Data6=reshape(Data6,size(Data6,1),size(Data6,2)*size(Data6,3)); % size(Data6)=[3 170280]; end %% Don't worry about the rest of this script. Just crap used for demo. function U=setUnionNoRep(a,b,c,d) % <--Not used. Usage depends on how you wish to collect sets and behavior to evoke U=union(a,union(b,union(c,d))); end function U=setUnion(a,b,c,d)% <--Not used. Usage depends on how you wish to collect sets etc. U=[a b c d]; end function A=xrandMod a=0.5; b=29.499999999999; A= a+(b-a)*rand(1,70); A=round(A); end function [S1, S2, S3, S4]=makeUpRandomCrap(N,showPrintOut) rng(4) % Seed the RNG to prevent fluctuation S1=[]; S2=[]; S3=[]; S4=[]; flds=[{'Elon'},{'Musk'},{'says'},{'he'},{'lost'},{'a'},{'multi'},{'billion'},{'dollar'},{'contract'},{'when'},{'SpaceX'},{'didnt'},{'hire'},{'the'},{'public'},{'official'}];%#ok<NOPRT> structNoms=[{'Credit'},{'ratings'},{'agencies'},{'just'},{'realized'},{'that'},{'climate'},{'change'},{'threat'},{'to'},{'world'},{'economy'} flds]; junk=unique(xrandMod) ; for i=1:17, for j=1:12, eval([structNoms{junk(i)} '.' flds{j} '=logical(rand(round(10*rand)));']); end; eval(['S1=[S1 ' structNoms{junk(i)} '];']); end; S2=S1; S3=S1; S4=S1; for i=1:18, S1(i).val=logical(randn(3,N)); S2(i).val=logical(randn(3,N)); S3(i).val=logical(randn(3,N)); S4(i).val=logical(randn(3,N)); end %% Print this out to com window to show what the internals of S1 looks like %% Modify vars called JUNKI, JUNKJ manually to see other fields etc. % junki shows values of S1(i), junkj shows values of S1.jthFieldName % 1<=junki<=18, 1<=junkj<=12. S1.val is the 13th field. if showPrintOut junki=ceil(18*rand); junkj=ceil(12*rand); fprintf('\t\t\t\t%s\n\n','This is what S1 looks like --'); disp(S1), fprintf('\n\t\t\t\t%s\n',['Here''s what S1(' num2str(junki) ') looks like --']); S1(junki),%#ok<NOPRT> fprintf('\n\t\t\t\t%s\n',['Here''s what ''S1.' flds{junkj} ''' looks like, where ''' flds{junkj} ''' is a field --']); for i=1:17, disp(['S1(' num2str(i) ').' flds{junkj} ]); eval(['disp(S1(' num2str(i) ').' flds{junkj} '),' ]), end; end end % % % Plot figure showing how D1Si indices overlap % % % Dataj is made of union( resampled_j(Si) )_i=1:4. Here resampled_j % % % represents the jth resampling mask on Si. So need to compare the six % % % sets-of-4 resampling masks. % % figure(1) % % mar='+.od'; % % col='rgbkcm'; % % for i=1:6,subplot(2,3,i), % % for j=1:4, plot(BigS{j,i}/1e4,[col(i) mar(j) '-'],'MarkerSize',2,'LineWidth',1.5), hold on, % % end; % % title(['Four index-sets whose union creates Data' num2str(i)],'FontSize',14), % % ylabel(['S/10000'],'FontSize',14), % % legend('S1','S2','S3','S4','location','SouthEast'); % % axis([0 4140 0 4.1]) % % end; % % % % % Plot figure showing how DjS1 indices overlap % % % Si gets resampled by BigS{i,j} 6 times to create components for % % % Dataj. Its useful to see the appearance of the 6 resampling vectors. % % figure(1) % % mar='+.ods^'; % % col='rgbkcm'; % % for i=1:4,subplot(2,2,i), % % for j=1:6, plot(BigS{i,j}/1e4,[col(i) mar(j) '-'],'MarkerSize',2,'LineWidth',1.5), hold on, % % end; % % title(['Four index-sets whose union creates Data' num2str(i)],'FontSize',14), % % ylabel(['S/10000'],'FontSize',14), % % legend('S1','S2','S3','S4','location','SouthEast'); % % axis([0 4140 0 4.1]) % % end; Version 2.b - Image based approachInsert the value of M in the following before running. It requires Matlab's image processing tool (specifically the function bwlabel). Line numbers 2-13 in the following replace lines 28-53 in the above script. close all; clear all; clc BW=~isnan(M); % Threshold N, BW image G=bwlabel(BW); % This could be written manually, if you don't have the image tool A=G; try for i=1:6, A(:,2*i-1:2*i)=A(:,2*i-1:2*i)*(i+20)*100 ; end; assert(length(unique( A(:) ))==25); catch, for i=1:6, A(:,2*i-1:2*i)=A(:,2*i-1:2*i)*(i+20*rand)*100 ; end; end r0=zeros(1,24); r1=r0; c0=r0; c1=r0; v=unique(A(:)); v=v(2:end); for i=1:24, [Iv, Jv]=ind2sub(size(A),find(A==v(i))); r0(i)=min(Iv); c0(i)=min(Jv); r1(i)=max(Iv); c1(i)=max(Jv); end; This is the output you should see when you run the function junk. %% OUTPUT IN COMMAND WINDOW DESCRIBING THE MADE UP S1/S2/S3/S4 struct arrays. This is what S1 looks like -- 1x18 struct array with fields: Elon Musk says he lost a multi billion dollar contract when SpaceX val Here's what S1(17) looks like -- ans = Elon: [4x4 logical] Musk: 1 says: 1 he: [3x3 logical] lost: 1 a: [8x8 logical] multi: [9x9 logical] billion: 1 dollar: [6x6 logical] contract: [2x2 logical] when: [9x9 logical] SpaceX: [7x7 logical] val: [3x42380 logical] Here's what 'S1.Musk' looks like, where 'Musk' is a field -- S1(1).Musk 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 S1(2).Musk 1 1 1 1 1 1 1 1 1 ... etc. You also need to visualize the rules contained in the M matrix. There are two ways to interpret the whole M-rules to fetch S-data thing - Rows of M codify resampling rules for each data source S. Columns of M codify components of S making up each Data. The visualization helps understand overlap or exclusivity in the sampling indices (rules). There are two plot scripts at the end of my main script. One generates the row picture, the other generates the column picture. Image below shows the column picture. Couple of things I see from the image: two distinct trends in the 4 sets of lines (representing 4 rows) per subplot. The trend originates because NaN width reduces numel in S3j/S4j as compared to S1j/S2j. There are some nice hysteresis structures in Data3-Data6 (for example Data6 plot shows 5 boxes - that's my hysteresis structure). Depending upon the actual experimental data (stochastic/periodic/periodic+non-stationary), you might get to see 5 to 4 subpopulation clusters.