ERROR:NestIOUtils:weights must be a 1-dimensional NumPy array

[nicolomeneghetti]

Hi all.

I was trying to convert a (very simple) custom-made bionet into a point neuron model.

I tried doing this by copying the network of the biophysical model into a new folder and modifying:

• in the node types csv I have changed the model type, the model template, and the dynamic parameters
• in the edge types csv I have changed the model template and the dynamic parameters

After having built the environment through the command:
from bmtk.utils.sim_setup import build_env_pointnet

build_env_pointnet(base_dir='.',
network_dir='network',
config_file='config.point.json',
tstop=3000.0,
dt=0.01,
current_clamp= {
'amp': 0.150, # Current size (pA)
'delay': 500, # Time from start of simulation to onset of step (ms)
'duration': 2000 # Duration of current step (ms)
},
include_examples=True, # Copies components files
)

I get this error message:

         -- N E S T --

Copyright (C) 2004 The NEST Initiative

Version: 3.3
Built: Aug 9 2022 10:44:45

This program is provided AS IS and comes with
NO WARRANTY. See the file LICENSE for details.

Problems or suggestions?
Visit https://www.nest-simulator.org

Type 'nest.help()' to find out more about NEST.

2022-10-18 22:08:40,086 [INFO] Created log file
INFO:NestIOUtils:Created log file
2022-10-18 22:08:40,106 [INFO] Batch processing nodes for l4/0.
INFO:NestIOUtils:Batch processing nodes for l4/0.
2022-10-18 22:08:40,110 [INFO] Batch processing nodes for l4/1.
INFO:NestIOUtils:Batch processing nodes for l4/1.
2022-10-18 22:08:40,120 [INFO] Setting up output directory
INFO:NestIOUtils:Setting up output directory
2022-10-18 22:08:40,122 [INFO] Building cells.
INFO:NestIOUtils:Building cells.
2022-10-18 22:08:40,128 [INFO] Building recurrent connections
INFO:NestIOUtils:Building recurrent connections
2022-10-18 22:08:40,136 [ERROR] weights must be a 1-dimensional NumPy array
ERROR:NestIOUtils:weights must be a 1-dimensional NumPy array
Traceback (most recent call last):

File "/home/nest/Desktop/bmtk github stuff/bmtk-develop/examples/nico_try_pointconv/run_pointnet.py", line 14, in
main('/home/nest/Desktop/bmtk github stuff/bmtk-develop/examples/nico_try_pointconv/config.l4point.json')

File "/home/nest/Desktop/bmtk github stuff/bmtk-develop/examples/nico_try_pointconv/run_pointnet.py", line 9, in main
sim = pointnet.PointSimulator.from_config(configure, network)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/pointnet/pointsimulator.py", line 243, in from_config
graph.build_recurrent_edges()

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/pointnet/pointnetwork.py", line 171, in build_recurrent_edges
self._nest_connect(nest_srcs, nest_trgs, conn_spec='one_to_one', syn_spec=edge.nest_params)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/pointnet/pointnetwork.py", line 227, in _nest_connect
nest.Connect(nest_srcs, nest_trgs, conn_spec=conn_spec, syn_spec=syn_spec)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/nest/ll_api.py", line 229, in stack_checker_func
return f(*args, **kwargs)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/nest/lib/hl_api_connections.py", line 257, in Connect
connect_arrays(pre, post, weights, delays, synapse_model, syn_param_keys, syn_param_values)

File "pynestkernel.pyx", line 300, in pynestkernel.NESTEngine.connect_arrays

TypeError: weights must be a 1-dimensional NumPy array

Can someone help me out?

Thank you in advance for any help.

[nicolomeneghetti]

I would also like to add the two network csv files:

1. node types:
   node_type_id rotation_angle_xaxis model_name dynamics_params model_type model_processing ei_type model_template
   100 0 Scnn1a 472363762_point.json point_process nan e nest:iaf_psc_alpha
   101 0 Rorb 472912177_point.json point_process nan e nest:iaf_psc_alpha
   102 0 Nr5a1 473862421_point.json point_process nan e nest:iaf_psc_alpha
   103 0 PValb 472912177_point.json point_process nan i nest:iaf_psc_alpha

2. edges types:
   edge_type_id target_query source_query model_template dynamics_params syn_weight delay
   100 ei_type=='e' ei_type=='e' static_synapse ExcToExc.json 0.2 2.0
   101 ei_type=='i' ei_type=='e' static_synapse InhToExc.json 0.2 2.0
   102 ei_type=='e' ei_type=='i' static_synapse ExcToInh.json 0.3 2.0
   103 ei_type=='i' ei_type=='i' static_synapse InhToInh.json 0.5 2.0

[xpliu16]

When building the network, what were the connection_rule and connection_params? In the edges HDF5 file, are nsyns defined for all connections?

[nicolomeneghetti]

Thanks first of all for your reply.

I have copied both the connection_rule and connection_params in some example code. Here follows the code that I have used for adding the edges in the biophysical detailed model (that is called l4):

def exc_exc_rule(source, target, max_syns):
"""Connection rule for exc-->exc neurons, should return an integer 0 or greater"""
if source['node_id'] == target['node_id']:
return 0
src_tuning = source['tuning_angle']
trg_tuning = target['tuning_angle']
tuning_dist = np.abs((src_tuning - trg_tuning + 180) % 360 - 180)
probs = 1.0 - (np.max((tuning_dist, 10.0)) / 180.0)
return np.random.binomial(n=max_syns, p=probs)

conns = l4.add_edges(
source=l4.nodes(ei_type='e'),
target=l4.nodes(ei_type='e'),
connection_rule=exc_exc_rule,
connection_params={'max_syns': 15},
syn_weight=3.0e-05,
delay=2.0,
dynamics_params='AMPA_ExcToExc.json',
model_template='Exp2Syn',
)

from bmtk.builder.bionet import rand_syn_locations

conns.add_properties(
['afferent_section_id', 'afferent_section_pos', 'afferent_swc_id', 'afferent_swc_pos'],
rule=rand_syn_locations,
rule_params={
'sections': ['basal', 'apical'],
'distance_range': [30.0, 150.0],
'morphology_dir': 'components/morphologies'
},
dtypes=[int, float, int, float]
)

def others_conn_rule(source, target, max_syns, max_distance=300.0, sigma=60.0):
if source['node_id'] == target['node_id']:
return 0
dist = np.sqrt((source['x'] - target['x'])**2 + (source['z'] - target['z'])**2)
if dist > max_distance:
return 0
prob = np.exp(-(dist/sigma)**2) # Gaussian fall off with distance
return np.random.binomial(n=max_syns, p=prob)

%%%%%%%%%%%%%%%
% Create e --> i connections
conns = l4.add_edges(
source=l4.nodes(ei_type='e'),
target=l4.nodes(ei_type='i'),
connection_rule=others_conn_rule,
connection_params={'max_syns': 8},
syn_weight=0.0006,
delay=2.0,
dynamics_params='AMPA_ExcToInh.json',
model_template='Exp2Syn',
)

conns.add_properties(
['afferent_section_id', 'afferent_section_pos', 'afferent_swc_id', 'afferent_swc_pos'],
rule=rand_syn_locations,
rule_params={
'sections': ['somatic', 'basal'],
'distance_range': [0.0, 1.0e+20],
'morphology_dir': 'components/morphologies'
},
dtypes=[int, float, int, float]
)

%%%%%%%%%%%%%%%%%%%%%%
% Create i --> e connections
conns = l4.add_edges(
source=l4.nodes(ei_type='i'),
target=l4.nodes(ei_type='e'),
connection_rule=others_conn_rule,
connection_params={'max_syns': 4},
syn_weight=0.07,
delay=2.0,
dynamics_params='GABA_InhToExc.json',
model_template='Exp2Syn',
)
conns.add_properties(
['afferent_section_id', 'afferent_section_pos', 'afferent_swc_id', 'afferent_swc_pos'],
rule=rand_syn_locations,
rule_params={
'sections': ['somatic', 'basal', 'apical'],
'distance_range': [0.0, 50.0],
'morphology_dir': 'components/morphologies'
},
dtypes=[int, float, int, float]
)
%%%%%%%%%%%%%%%%%%%%%%%%%
%Create i --> i connections
conns = l4.add_edges(
source=l4.nodes(ei_type='i'),
target=l4.nodes(ei_type='i'),
connection_rule=others_conn_rule,
connection_params={'max_syns': 4},
syn_weight=0.00015,
delay=2.0,
dynamics_params='GABA_InhToInh.json',
model_template='Exp2Syn',
)
conns.add_properties(
['afferent_section_id', 'afferent_section_pos', 'afferent_swc_id', 'afferent_swc_pos'],
rule=rand_syn_locations,
rule_params={
'sections': ['somatic', 'basal', 'apical'],
'distance_range': [0.0, 1.0e+20],
'morphology_dir': 'components/morphologies'
},
dtypes=[int, float, int, float]
)
l4.build()
l4.save(output_dir='network')`

%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%

As far as it concerns the edges HDF5 file I can't see the nsyns in the keys.
In "edges"->"l4_to_l4"->"0" there are four fields: (1)afferent_swc_id, (2)afferent_swc_pos, (3)afferent_section_id, (4)afferent_section_pos. Hence no reference to nsyns.
I know that nsyns pops up when building pointnet networks (even though I am not sure what's inside of nsyns). Is the lack of nsyns the problem? If yes how can I therefore convert a bionet network into a poinet? Should I also make modifications (and how) to the h5 edge files?

Thanks again in advance for any help.

Best

[xpliu16]

Thanks for bringing this issue to our attention - we think it is an interaction between conversion, add_properties, and perhaps also Nest version. BMTK will store synapse weights as a single number if it is given as such (to be apply to all synapses). Nest, at least the Nest 3 versions, seem to only allow an array to be received. I believe normally, this isn't an issue because nsyns is different for the different edges and forces the total weight = weight * nsyns to be an array. Add_properties gives each individual synapse an entry and we lose nsyns.

We've merged a little fix. Try pulling from the repo and rerunning python setup.py install. Let us know if that fixes it for you?

[nicolomeneghetti]

Thanks for the reply. One last question (before opening another issue but of different nature D: ): can I update this fix by running again the conda installation?

Thanks in adavce,

Best

[xpliu16]

Kael just put up a new release and updated anaconda. Can you try that and see if it fixes the issue for you?

[nicolomeneghetti]

Dear alls,

thank you very much. This problem has been solved.

Concurrently, however, I am now facing a new issue when trying using bionet. For example I tried running the "bio_14cells" example and I got this neuron error:

Python 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:23:11)
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

runfile('/home/nest/Desktop/bmtk github stuff/bmtk-develop/examples/bio_14cells/run_bionet.py', wdir='/home/nest/Desktop/bmtk github stuff/bmtk-develop/examples/bio_14cells')
2022-11-02 14:37:31,663 [INFO] Created log file
2022-11-02 14:37:31,783 [INFO] Building cells.
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 4: could not parse: 1 1 303.16 379.4648 28.56 5.4428 -1
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 5: could not parse: 2 3 302.6646 375.232 23.2562 0.2524 1
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 6: could not parse: 3 3 302.469 374.2356 21.9996 0.2615 2
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 7: could not parse: 4 3 302.0057 373.2918 21.0081 0.3026 3
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 8: could not parse: 5 3 301.2358 372.4956 20.7318 0.36 4
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 9: could not parse: 6 3 300.3366 371.832 20.7847 0.3728 5
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 10: could not parse: 7 3 299.7177 370.8768 20.9602 0.3575 6

[....]

error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 3781: could not parse: 3778 3 196.7028 334.8694 12.9396 0.2089 3777
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 3782: could not parse: 3779 3 195.7636 334.8191 12.9847 0.1992 3778
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 3783: could not parse: 3780 3 195.5027 334.8431 11.3277 0.1794 3779
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 3784: could not parse: 3781 3 194.8655 334.2619 10.9175 0.1261 3780
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 3785: could not parse: 3782 3 194.1711 333.5263 11.3341 0.2603 3781
error ./../bio_components/morphologies/Scnn1a_473845048_m.swc line 3786: could not parse: 3783 3 194.1368 332.904 12.04 0.1144 3782
NEURON: Arg out of range in user function
near line 0
^
Import3d_Section[0].Matrix(3, 0)
Import3d_Section[0].init(0, 0)
Import3d_SWC_read[0].mksection(0, 0, 0)
Import3d_SWC_read[0].mksections()
and others
Traceback (most recent call last):

File "/home/nest/Desktop/bmtk github stuff/bmtk-develop/examples/bio_14cells/run_bionet.py", line 32, in
run(config_path)

File "/home/nest/Desktop/bmtk github stuff/bmtk-develop/examples/bio_14cells/run_bionet.py", line 12, in run
sim = bionet.BioSimulator.from_config(conf, network=graph)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/bionet/biosimulator.py", line 402, in from_config
network.build_nodes()

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/bionet/bionetwork.py", line 178, in build_nodes
cell = self._build_cell(bionode=node, population_name=node_pop.name)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/bionet/bionetwork.py", line 161, in _build_cell
cell = self._model_type_map[bionode.model_type](bionode, population_name=population_name, bionetwork=self)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/bionet/biocell.py", line 75, in init
super(BioCell, self).init(node=node, population_name=population_name, network=bionetwork)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/bionet/cell.py", line 54, in init
self._hobj = node.load_cell()

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/bionet/sonata_adaptors.py", line 70, in load_cell
hobj = cell_fnc(self, template_name, dynamics_params)

File "/home/nest/anaconda3/envs/allen_inst_bkmt/lib/python3.9/site-packages/bmtk/simulator/bionet/default_setters/cell_models.py", line 75, in Biophys1
hobj = h.Biophys1(str(morphology_file))

RuntimeError: hoc error

I have also tried running again nrnivmodl with no sucess unfortunately. By the way, if this is a new unrelated issue I will open another issue post here so that you can close the current one as solved.

Thanks again for your help,

Best regards,

Nicolò

[nicolomeneghetti]

Dear all,

I have solved the error mentioned in the previous comment. I leave here the solution in case anyone else would need.

The "RuntimeError: hoc error" mentioned above seems to be due to a change I made in Spyder's preferences. The error indeed occured when I set the 'Graphics backend' to 'Automatic' instead of the (default) 'Inline' option.

By setting again the Inline option and restarting a new spyder console solved the problem for me. Honestly I am not sure why this fix works , yet it does....

Thanks again for your help,

Best regards,

Nicolò

[xpliu16]

Glad you were able to resolve that and really appreciate you sharing the solution!
Should we close this issue now?

[nicolomeneghetti]

Yes, the problems were solved with the new release.

Thank again for your help,

Best regards,

Nicolò