Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce memory footprint of configuration parser #1102

Closed
frankmorgner opened this issue Jul 18, 2017 · 0 comments
Closed

Reduce memory footprint of configuration parser #1102

frankmorgner opened this issue Jul 18, 2017 · 0 comments
Labels
enhancement

Comments

@frankmorgner
Copy link
Member

Expected behaviour

Reasonable memory consumption.

Actual behaviour

pkcs11-tool --test --login takes about 400 KB of memory, of which most memory is consumed by scconf_parse() and friends (via strdup()).

Steps to reproduce

  1. valgrind --tool=massif pkcs11-tool --test --login
  2. ms_print massif.out.*

Logs

Command:            pkcs11-tool --test --login
Massif arguments:   (none)
ms_print arguments: massif.out.11963
--------------------------------------------------------------------------------


    KB
382.1^                                                   #                    
     |                                                   #                    
     |                                                   #                    
     |                                                   #                    
     |                                                   #        :           
     |                                                   #        :           
     |                                                   #        :           
     |           ::::::::::::::@@@@@@::::::::::::@:::::::#::::@:::::@:::::@:::
     |         :@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |         :@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |        @:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |       @@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |       @@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |      @@@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |     @@@@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |    @@@@@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |   @@@@@@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |  :@@@@@@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     | ::@@@@@@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
     |:::@@@@@@:@::     ::    :@     ::    ::    @:    ::#::::@:::::@:::::@:::
   0 +----------------------------------------------------------------------->Mi
     0                                                                   81.00

Number of snapshots: 88
 Detailed snapshots: [9, 10, 11, 13, 14, 17, 21, 22, 23, 26, 36, 42, 49, 50 (peak), 60, 70, 80]

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
  0              0                0                0             0            0
  1        563,825               40               32             8            0
  2        843,477           20,424           15,370         5,054            0
  3      1,259,730           27,664           18,627         9,037            0
  4      1,819,084           42,608           30,514        12,094            0
  5      2,171,625           49,392           35,409        13,983            0
  6      2,651,574           59,040           42,380        16,660            0
  7      3,074,055           65,752           47,441        18,311            0
  8      3,720,088           78,352           55,863        22,489            0
  9      4,295,332           87,704           62,179        25,525            0
70.90% (62,179B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->21.00% (18,416B) 0x5B05388: strdup (strdup.c:42)
| ->14.34% (12,580B) 0x53E2DFA: scconf_parse_token (parse.c:266)
...
>19.12% (16,768B) 0x53E280D: scconf_item_add_internal (parse.c:110)
| ->14.67% (12,864B) 0x53E2DD8: scconf_parse_token (parse.c:262)
...
->17.24% (15,120B) 0x4E961C6: CRYPTO_malloc (mem.c:346)
| ->07.47% (6,552B) 0x4F11624: lh_insert (lhash.c:191)
...
@frankmorgner frankmorgner mentioned this issue Jul 20, 2017
Closed
@frankmorgner frankmorgner mentioned this issue Aug 4, 2017
Merged
frankmorgner added a commit to frankmorgner/OpenSC that referenced this issue Jul 4, 2018
@frankmorgner
Distribute a minimal opensc.conf
3a95d7c 
closes OpenSC#1102
frankmorgner added a commit that referenced this issue Jul 11, 2018
@frankmorgner
Distribute a minimal opensc.conf
1cf4ed1 
closes #1102
frankmorgner added a commit to frankmorgner/OpenSC that referenced this issue Jul 11, 2018
@frankmorgner
Distribute a minimal opensc.conf
f519144 
closes OpenSC#1102
frankmorgner added a commit to frankmorgner/OpenSC that referenced this issue Jul 11, 2018
@frankmorgner
Distribute a minimal opensc.conf
053bea4 
closes OpenSC#1102
frankmorgner added a commit to frankmorgner/OpenSC that referenced this issue Jul 12, 2018
@frankmorgner
Distribute a minimal opensc.conf
7d73591 
closes OpenSC#1102
mouse07410 added a commit to mouse07410/OpenSC that referenced this issue Aug 7, 2018
@mouse07410
Restore DEFAULT_PKCS11_PROVIDER, fixing the OpenSC#1102 debacle (losi…
0d9a0b3 
…ng default PKCS11 library on MacOS)
@dengert dengert mentioned this issue Aug 15, 2018
Closed
dengert added a commit to dengert/OpenSC that referenced this issue Aug 20, 2018
@dengert
Reduced memory footprint of configuration processor - improvement Ope…
3c89dff 
…nSC#1102

The scconf parser is changes to discard comments and blocks that are empty.
Previously comments were saved in memory as well as empty blocks.
Traditionally opensc.conf files have add many lines commented out or served
as a man page for opensc.conf.

c003f38 also addressed issue OpenSC#1102 by
removing all but of a few lines to create a minimal opensc.conf file and
an opensc.conf.example file. With this change a user could take the example
opensc.conf with comments and use it as the opensc.conf as they have done
on the past.

c003f38 could be reverted so users would
not see a major configuration change in the next release and still get the
benefit of a small opensc.conf memory footprint.

An opensc.conf file has been about 48K bytes, with 40K being comments.

 On branch jakuje-x41sec-merge
 Your branch is up-to-date with 'jakuje/x41sec-merge'.
 Changes to be committed:
	modified:   src/scconf/parse.c
@dengert dengert mentioned this issue Aug 20, 2018
Closed
5 tasks
@dengert dengert mentioned this issue Nov 24, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@frankmorgner